-- Logs begin at Mon 2022-10-31 11:55:10 UTC, end at Mon 2022-10-31 12:49:53 UTC. -- Oct 31 12:19:16.684166 dv-cinder-796170-6-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Oct 31 12:19:16.692451 dv-cinder-796170-6-iscsi devstack@keystone.service[81479]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Oct 31 12:19:16.695003 dv-cinder-796170-6-iscsi devstack@keystone.service[81479]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Oct 31 12:19:16 2022] *** Oct 31 12:19:16.695138 dv-cinder-796170-6-iscsi devstack@keystone.service[81479]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Oct 31 12:19:16.695265 dv-cinder-796170-6-iscsi devstack@keystone.service[81479]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Oct 31 12:19:16.695381 dv-cinder-796170-6-iscsi devstack@keystone.service[81479]: nodename: dv-cinder-796170-6-iscsi Oct 31 12:19:16.695476 dv-cinder-796170-6-iscsi devstack@keystone.service[81479]: machine: x86_64 Oct 31 12:19:16.695579 dv-cinder-796170-6-iscsi devstack@keystone.service[81479]: clock source: unix Oct 31 12:19:16.695678 dv-cinder-796170-6-iscsi devstack@keystone.service[81479]: pcre jit disabled Oct 31 12:19:16.695776 dv-cinder-796170-6-iscsi devstack@keystone.service[81479]: detected number of CPU cores: 4 Oct 31 12:19:16.695873 dv-cinder-796170-6-iscsi devstack@keystone.service[81479]: current working directory: / Oct 31 12:19:16.695971 dv-cinder-796170-6-iscsi devstack@keystone.service[81479]: detected binary path: /usr/bin/uwsgi-core Oct 31 12:19:16.696081 dv-cinder-796170-6-iscsi devstack@keystone.service[81479]: your processes number limit is 47680 Oct 31 12:19:16.696176 dv-cinder-796170-6-iscsi devstack@keystone.service[81479]: your memory page size is 4096 bytes Oct 31 12:19:16.696276 dv-cinder-796170-6-iscsi devstack@keystone.service[81479]: detected max file descriptor number: 2048 Oct 31 12:19:16.696378 dv-cinder-796170-6-iscsi devstack@keystone.service[81479]: lock engine: pthread robust mutexes Oct 31 12:19:16.696631 dv-cinder-796170-6-iscsi devstack@keystone.service[81479]: thunder lock: enabled Oct 31 12:19:16.697496 dv-cinder-796170-6-iscsi devstack@keystone.service[81479]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Oct 31 12:19:16.697678 dv-cinder-796170-6-iscsi devstack@keystone.service[81479]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Oct 31 12:19:16.736723 dv-cinder-796170-6-iscsi devstack@keystone.service[81479]: Python main interpreter initialized at 0x55b7741f79c0 Oct 31 12:19:16.736723 dv-cinder-796170-6-iscsi devstack@keystone.service[81479]: python threads support enabled Oct 31 12:19:16.736723 dv-cinder-796170-6-iscsi devstack@keystone.service[81479]: your server socket listen backlog is limited to 100 connections Oct 31 12:19:16.736723 dv-cinder-796170-6-iscsi devstack@keystone.service[81479]: your mercy for graceful operations on workers is 90 seconds Oct 31 12:19:16.737060 dv-cinder-796170-6-iscsi devstack@keystone.service[81479]: mapped 403077 bytes (393 KB) for 2 cores Oct 31 12:19:16.737151 dv-cinder-796170-6-iscsi devstack@keystone.service[81479]: *** Operational MODE: preforking *** Oct 31 12:19:16.737396 dv-cinder-796170-6-iscsi devstack@keystone.service[81479]: *** uWSGI is running in multiple interpreter mode *** Oct 31 12:19:16.737396 dv-cinder-796170-6-iscsi devstack@keystone.service[81479]: spawned uWSGI master process (pid: 81479) Oct 31 12:19:16.737596 dv-cinder-796170-6-iscsi systemd[1]: Started Devstack devstack@keystone.service. Oct 31 12:19:16.738278 dv-cinder-796170-6-iscsi devstack@keystone.service[81479]: spawned uWSGI worker 1 (pid: 81491, cores: 1) Oct 31 12:19:16.738442 dv-cinder-796170-6-iscsi devstack@keystone.service[81479]: spawned uWSGI worker 2 (pid: 81492, cores: 1) Oct 31 12:19:16.738602 dv-cinder-796170-6-iscsi devstack@keystone.service[81479]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Oct 31 12:19:21.041383 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 31 12:19:21.041383 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: from cryptography.utils import int_from_bytes Oct 31 12:19:21.042053 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 31 12:19:21.042053 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: from cryptography.utils import int_from_bytes Oct 31 12:19:21.042548 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 31 12:19:21.042548 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: from cryptography.utils import int_from_bytes Oct 31 12:19:21.042987 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 31 12:19:21.042987 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: from cryptography.utils import int_from_bytes Oct 31 12:19:21.091661 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 31 12:19:21.091860 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 31 12:19:21.092052 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] command line args: [] {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 31 12:19:21.092216 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 31 12:19:21.092325 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 31 12:19:21.092403 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 31 12:19:21.092507 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] ================================================================================ {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 31 12:19:21.092595 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] command line args: [] {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 31 12:19:21.092763 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] admin_token = **** {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 31 12:19:21.092847 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 31 12:19:21.093014 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ================================================================================ {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 31 12:19:21.093093 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] config_dir = [] {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 31 12:19:21.093203 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 31 12:19:21.093278 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] admin_token = **** {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 31 12:19:21.093389 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] config_source = [] {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 31 12:19:21.093508 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] config_dir = [] {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 31 12:19:21.093587 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] debug = True {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 31 12:19:21.093700 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 31 12:19:21.093784 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 31 12:19:21.094010 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] config_source = [] {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 31 12:19:21.094071 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] debug = True {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 31 12:19:21.094132 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 31 12:19:21.094255 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 31 12:19:21.094360 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 31 12:19:21.094487 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 31 12:19:21.094573 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 31 12:19:21.094672 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 31 12:19:21.094744 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 31 12:19:21.094916 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 31 12:19:21.094993 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] list_limit = None {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 31 12:19:21.095148 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] log_config_append = None {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 31 12:19:21.095257 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 31 12:19:21.095365 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 31 12:19:21.095439 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] list_limit = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 31 12:19:21.095537 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] log_dir = None {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 31 12:19:21.095613 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] log_config_append = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 31 12:19:21.095714 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] log_file = None {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 31 12:19:21.095789 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 31 12:19:21.095914 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] log_dir = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 31 12:19:21.095995 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 31 12:19:21.096113 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] log_file = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 31 12:19:21.096190 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 31 12:19:21.096325 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 31 12:19:21.096407 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 31 12:19:21.096574 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 31 12:19:21.096690 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 31 12:19:21.096814 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 31 12:19:21.096916 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 31 12:19:21.097029 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 31 12:19:21.097142 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 31 12:19:21.097259 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 31 12:19:21.097382 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 31 12:19:21.097382 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 31 12:19:21.097599 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 31 12:19:21.097715 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 31 12:19:21.097832 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 31 12:19:21.097954 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 31 12:19:21.098077 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 31 12:19:21.098188 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 31 12:19:21.098302 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 31 12:19:21.098419 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 31 12:19:21.098529 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 31 12:19:21.098643 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 31 12:19:21.098761 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 31 12:19:21.098872 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 31 12:19:21.098996 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 31 12:19:21.099108 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 31 12:19:21.099240 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 31 12:19:21.099354 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 31 12:19:21.099476 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 31 12:19:21.099578 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.15/identity {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 31 12:19:21.099688 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] publish_errors = False {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 31 12:19:21.099914 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 31 12:19:21.100071 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.15/identity {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 31 12:19:21.100187 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] publish_errors = False {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 31 12:19:21.100297 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 31 12:19:21.100297 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 31 12:19:21.100509 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 31 12:19:21.100625 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 31 12:19:21.100725 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 31 12:19:21.100725 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 31 12:19:21.100946 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 31 12:19:21.101066 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 31 12:19:21.101183 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] standard_threads = False {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 31 12:19:21.101294 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 31 12:19:21.101403 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 31 12:19:21.101514 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] standard_threads = False {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 31 12:19:21.101635 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 31 12:19:21.101635 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 31 12:19:21.101843 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 31 12:19:21.101958 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] use_journal = False {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 31 12:19:21.102075 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 31 12:19:21.102186 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] use_json = False {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 31 12:19:21.102347 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 31 12:19:21.102467 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] use_stderr = False {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 31 12:19:21.102580 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] use_journal = False {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 31 12:19:21.102690 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] use_syslog = False {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 31 12:19:21.102805 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] use_json = False {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 31 12:19:21.102917 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 31 12:19:21.103035 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] use_stderr = False {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 31 12:19:21.103035 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] use_syslog = False {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 31 12:19:21.103905 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.104020 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 31 12:19:21.104131 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.104253 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.104376 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.104376 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.104597 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.104597 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.104806 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.104923 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.105046 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.105164 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.105275 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.105384 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.105494 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.105603 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.105722 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.105838 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.106207 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.106327 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.106438 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.106547 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.106668 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.106782 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.106892 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.107002 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.107111 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.107238 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.107370 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.107493 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.107605 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.107716 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.107827 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.107943 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.108066 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.108178 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.108294 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.108405 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.108521 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.108638 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.108828 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.108943 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.109059 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.109176 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.109287 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.109405 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.109517 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.109627 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.109737 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.109847 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.109959 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.110095 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.110209 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.110319 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.110429 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.110539 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.110655 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.110773 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.110890 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.111003 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.111112 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.111265 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.111387 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.111387 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.111611 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.111611 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.111824 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.111936 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.112052 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.112173 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.112291 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.112402 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.112512 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.112621 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.112737 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.112737 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.112954 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.113065 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.113175 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.113290 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.113401 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.113518 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.113630 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.113739 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.113849 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.113958 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.114068 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.114252 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.114371 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.114371 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.114586 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.114586 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.114806 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.114925 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.115046 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.115156 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.115281 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.115391 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.115501 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.115623 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.115735 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.115848 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.115957 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.116051 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.116142 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.116246 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.116341 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.116432 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.116529 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.116622 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.116714 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.116814 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.116908 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.117001 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.117145 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.117270 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.117394 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.117506 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.117617 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.117617 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.117828 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.117828 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.118044 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.118155 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.118272 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.118272 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.118494 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.118606 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.118728 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.118728 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.118996 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.119110 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.119265 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.119378 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.119480 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.119480 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.119707 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.119820 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.119931 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.120048 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.120160 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.120270 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.120388 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.120502 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.120613 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.120737 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.120850 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.120960 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.121078 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.121191 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.121308 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.121410 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.121520 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.121631 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.121750 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.121863 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.121974 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.122085 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.122196 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.122306 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.122425 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.122540 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.122651 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.122763 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.122873 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.122873 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.123098 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.123233 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.123349 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.123460 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.123578 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.123578 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.123796 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.123909 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.124020 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.124131 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.124232 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.124341 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.124460 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.124572 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.124684 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.124794 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.124905 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.125016 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.125197 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.125318 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.125431 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.125541 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.125653 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.125653 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.125887 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.126008 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.126122 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.126304 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.126419 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.126531 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.126642 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.126762 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.126876 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.126987 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.127099 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.127099 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.127349 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.127469 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.127588 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.127701 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.127813 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.127924 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.128035 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.128153 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.128272 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.128384 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.128507 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.128620 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.128757 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] auth.external = None {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.128757 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.128985 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.128985 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] auth.external = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.129209 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.129322 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.129433 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.129433 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] auth.password = None {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.129656 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.129778 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] auth.token = None {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.129893 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.130006 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.130118 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] auth.password = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.130118 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] auth.token = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.130328 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.130452 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.130566 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.130679 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.130791 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.130791 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.131004 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.131004 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.131254 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.131254 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.131467 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.131580 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] credential.caching = True {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.131692 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.131815 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.131930 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.131930 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] credential.caching = True {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.132132 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.132243 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.132355 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.132475 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.132589 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.132700 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.132812 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.132812 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.133037 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.133162 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.133276 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.133276 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.133486 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.133486 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.133702 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.133825 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.133939 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.133939 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.134151 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.134263 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.134375 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.134993 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.135111 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.135272 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.135385 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.135385 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.135596 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.135726 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] federation.caching = True {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.135840 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.135952 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.136064 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.136176 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.136291 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] federation.caching = True {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.136411 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.136525 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.136637 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.136749 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.136861 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.136984 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.137099 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.137212 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.137324 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.137436 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.137548 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.137668 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.137782 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.137894 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.138007 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.138118 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.138230 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.138350 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.138471 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] identity.caching = True {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.138583 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.138695 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.138807 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.138919 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.139110 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] identity.caching = True {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.139235 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.139350 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.139463 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.139575 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.139688 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.139812 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.139926 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.140039 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.140152 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.140265 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.140379 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.140499 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.140880 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.140994 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.141106 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.141218 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.141330 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.141452 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.141566 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.141678 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.141791 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.141903 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.142020 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.142145 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.142259 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.142371 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.142484 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.142597 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.142709 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.142833 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.142947 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.143060 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.143191 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.143306 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.143429 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.143552 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.143666 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.143785 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.143898 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.144026 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.144138 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.144258 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.144372 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.144484 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.144597 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.144708 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.144819 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.144939 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.145052 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.145165 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.145277 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.145388 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.145500 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.145621 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.145736 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.145848 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.145960 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.146072 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.146183 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.146307 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.146421 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.146533 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.146645 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.146763 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.146884 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.146998 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.147110 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.147290 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.147403 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.147515 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.147639 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.147755 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.147755 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.147966 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.148078 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.148191 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.148311 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.148425 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.148536 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.148648 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.148648 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.148859 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.148982 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.149095 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.149207 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.149319 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.149431 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.149543 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.149662 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.149778 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.149890 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.150002 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.150958 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.151083 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.151243 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.151361 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.151481 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] ldap.user = None {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.151596 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.151709 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.151822 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.151822 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.user = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.152044 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.152044 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.152257 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.152368 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.152480 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.152600 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.152715 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.152826 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.152940 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.153052 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.153165 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.153285 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.153399 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.153511 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.153624 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.153736 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.153849 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.153970 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.154084 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.154197 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.154309 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.154420 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.154532 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.154652 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.154766 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.154879 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.154992 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.155104 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.155251 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.155377 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.155492 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.155604 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.155716 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.155828 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.155940 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.156061 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.156176 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.156288 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.156401 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.156512 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.156625 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.156745 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.156859 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.156971 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.157087 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.157199 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.157312 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.157432 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.157546 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.157658 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.157771 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.157883 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.157995 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.158115 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.158230 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.158349 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.158464 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.158576 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.158688 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.158808 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.158922 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.159034 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.159147 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.159280 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.159394 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.159517 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.159631 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.159744 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.159856 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.159968 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] resource.caching = True {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.160080 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.160200 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.160316 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.160428 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.160540 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] resource.caching = True {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.160652 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.160770 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.160891 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.161005 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.161117 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.161229 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.161342 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.161454 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.161574 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.161688 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.161799 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.161912 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.162023 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.162205 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.162321 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] role.caching = True {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.162435 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.162547 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] role.driver = sql {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.162659 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.162771 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.162891 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] role.caching = True {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.163006 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.163118 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] role.driver = sql {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.163248 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.163362 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.163475 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.163599 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.163713 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.163826 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.163938 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.164056 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.164169 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.164289 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.164404 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.164517 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.164630 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.164742 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.164853 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.164973 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.165088 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.165200 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.165312 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.165495 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.165612 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.165820 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.165941 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.166046 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.166046 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.166266 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.166369 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.166517 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.166635 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.166754 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.166869 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.166981 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.167096 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.167270 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.167386 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.167499 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.167613 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.167726 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.167847 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.167961 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.168074 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.168186 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.168299 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.168412 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.168532 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.168649 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.168649 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.168862 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.168974 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.169087 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.169087 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.169308 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.169421 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.169533 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.169645 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.169758 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.169879 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.169992 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.169992 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.170204 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.170316 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.170428 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.170549 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] token.caching = True {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.170663 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.170776 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.170889 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.170889 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] token.caching = True {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.171101 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.171240 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.171357 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.171469 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.171582 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.171694 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.171808 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.171928 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.172043 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.172155 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.172267 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.172380 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.172492 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.172611 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.172725 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.172838 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.172951 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.173064 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.173175 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.173175 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.173396 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.173396 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.173608 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.173720 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.173832 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.173953 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.174067 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.174067 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.174295 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.174295 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.174516 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.174638 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.174753 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] database.connection = **** {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.174866 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.174989 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.175103 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] database.connection = **** {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.175327 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.175446 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.175560 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.175560 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.175784 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.175900 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.176014 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.176126 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.176239 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.176352 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.176472 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.176586 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.176699 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.176811 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.176924 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.177041 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.177163 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.177277 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.177390 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.177502 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.177615 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.177727 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.177848 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.177962 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.178077 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.178189 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.178302 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.178414 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.178554 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.178669 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.178782 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.178894 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.179006 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.179119 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 31 12:19:21.179259 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.179259 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 31 12:19:21.179259 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 31 12:19:21.208143 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81492) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 31 12:19:21.208371 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81492) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 31 12:19:21.208598 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81492) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 31 12:19:21.208680 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 31 12:19:21.208865 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 31 12:19:21.209091 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 31 12:19:21.217463 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81492) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 31 12:19:21.218153 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 31 12:19:21.246474 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81492) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 31 12:19:21.246615 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81492) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 31 12:19:21.247436 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 31 12:19:21.247671 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 31 12:19:21.299418 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81492) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 31 12:19:21.299608 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 31 12:19:21.348185 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81492) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 31 12:19:21.348421 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81492) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 31 12:19:21.349009 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 31 12:19:21.349246 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 31 12:19:21.504972 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 31 12:19:21.505162 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81492) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 31 12:19:21.505244 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 31 12:19:21.505321 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81492) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 31 12:19:21.505394 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 31 12:19:21.505488 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81492) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 31 12:19:21.505611 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 31 12:19:21.505691 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81492) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 31 12:19:21.505832 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 31 12:19:21.505913 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81492) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 31 12:19:21.506051 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 31 12:19:21.506133 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81492) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 31 12:19:21.529456 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81492) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 31 12:19:21.529687 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81492) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 31 12:19:21.529875 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 31 12:19:21.529959 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81492) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 31 12:19:21.530114 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 31 12:19:21.530196 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81492) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 31 12:19:21.530332 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81492) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 31 12:19:21.530412 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 31 12:19:21.530551 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81492) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 31 12:19:21.530641 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 31 12:19:21.530764 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81492) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 31 12:19:21.530845 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 31 12:19:21.530975 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 31 12:19:21.531216 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 31 12:19:21.541742 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81492) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 31 12:19:21.542453 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 31 12:19:21.554411 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81492) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 31 12:19:21.554598 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81492) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 31 12:19:21.554819 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81492) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 31 12:19:21.554947 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 31 12:19:21.555029 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81492) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 31 12:19:21.555272 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81492) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 31 12:19:21.555353 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 31 12:19:21.555490 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81492) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 31 12:19:21.555571 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 31 12:19:21.555705 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81492) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 31 12:19:21.555787 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 31 12:19:21.555923 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 31 12:19:21.556141 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 31 12:19:21.556886 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 31 12:19:21.572837 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.573042 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 31 12:19:21.573384 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.573587 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 31 12:19:21.573864 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.573941 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.574109 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 31 12:19:21.574188 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.574376 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.574456 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.574637 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 31 12:19:21.574889 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.574969 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.575183 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.575455 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.575972 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.584277 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.584559 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.584829 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.585113 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.585403 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.585684 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.585958 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.586245 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.592851 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81492) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 31 12:19:21.593806 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81491) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 31 12:19:21.595755 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81492) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 31 12:19:21.596039 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.596322 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.596607 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.596761 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81491) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 31 12:19:21.596892 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.597047 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.597159 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.597331 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.597431 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.597616 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.597693 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.597901 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.598080 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.598158 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.598357 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.598435 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.598629 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.598724 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.599059 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.599370 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.599647 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.622201 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.623328 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.623685 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81492) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 31 12:19:21.623974 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.624801 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81491) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 31 12:19:21.625089 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.630294 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81492) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 31 12:19:21.630583 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.630866 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.631501 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81491) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 31 12:19:21.631791 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.632077 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.637468 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81492) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 31 12:19:21.637759 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.638689 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81491) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 31 12:19:21.638979 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.642195 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81492) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 31 12:19:21.642487 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.642773 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.643059 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.643367 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.643470 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81491) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 31 12:19:21.643674 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.643755 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.643958 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.644046 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.644232 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.644319 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.644573 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.644875 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.645154 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.645436 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.660395 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.660670 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.661766 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.662050 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.664554 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81492) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 31 12:19:21.666659 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81491) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 31 12:19:21.668404 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.668702 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.670105 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.670406 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.672966 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81492) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 31 12:19:21.674692 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81491) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 31 12:19:21.676153 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81492) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 31 12:19:21.677876 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81491) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 31 12:19:21.679397 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.679698 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.679990 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.680283 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.680569 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.680858 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.681127 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.681465 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.681920 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.682216 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.682514 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.682807 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.696765 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81492) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 31 12:19:21.697079 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.697348 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.697612 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.698806 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81491) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 31 12:19:21.699095 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.699390 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.699658 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.704518 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.706060 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.706562 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.707638 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.707917 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.708150 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.709707 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.709987 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.710572 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81492) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 31 12:19:21.710859 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.711277 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.711568 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.711863 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.713180 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81491) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 31 12:19:21.713479 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.713872 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.714162 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.714457 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.723667 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81492) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 31 12:19:21.723969 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.724350 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.724637 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.724930 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.725218 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.725499 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.725770 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81491) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 31 12:19:21.726066 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.726448 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.726750 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.727025 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.727341 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.727624 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.741174 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81492) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 31 12:19:21.743342 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81491) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 31 12:19:21.744797 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81492) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 31 12:19:21.746931 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81491) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 31 12:19:21.747729 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.749163 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.749963 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.750783 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81492) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 31 12:19:21.751068 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.751376 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.751458 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.753054 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81491) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 31 12:19:21.753341 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.753628 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.757742 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.759241 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81492) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 31 12:19:21.760147 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.761645 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81491) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 31 12:19:21.762469 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.762760 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.763047 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.763353 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.764715 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.765010 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.765299 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.765831 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.771469 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81492) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 31 12:19:21.771762 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.772043 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.773750 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81491) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 31 12:19:21.774046 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.774331 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.780633 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81492) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 31 12:19:21.780922 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.781205 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.781483 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.781758 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.782050 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.782330 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.782641 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.782777 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81491) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 31 12:19:21.782953 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.783071 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.783281 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.783388 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.783605 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.783695 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.783889 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.783970 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.784160 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.784241 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.784425 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.784504 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.784743 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.785015 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.785290 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.785604 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.785903 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.786175 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.786448 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 31 12:19:21.818402 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55b7741f79c0 pid: 81492 (default app) Oct 31 12:19:21.820213 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55b7741f79c0 pid: 81491 (default app) Oct 31 12:19:21.821601 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcc3e4e-6901-4031-b413-a2213b281468 None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:21.821889 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcc3e4e-6901-4031-b413-a2213b281468 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:21.822213 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcc3e4e-6901-4031-b413-a2213b281468 None None] PATH_INFO: `/v3` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:21.826392 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1/1] 10.222.0.15 () {50 vars in 791 bytes} [Mon Oct 31 12:19:21 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:19:28.604645 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9daa74-0bf2-4c6d-90c2-3a4e91f45065 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:28.605164 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9daa74-0bf2-4c6d-90c2-3a4e91f45065 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:28.605164 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9daa74-0bf2-4c6d-90c2-3a4e91f45065 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:28.609431 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1/2] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:19:28 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 31 12:19:28.614524 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612799c5-5f43-4c6e-9a8d-2ed8022a4354 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:28.615104 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612799c5-5f43-4c6e-9a8d-2ed8022a4354 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:28.615104 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612799c5-5f43-4c6e-9a8d-2ed8022a4354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:28.686625 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: INFO dbcounter [None req-612799c5-5f43-4c6e-9a8d-2ed8022a4354 None None] Registered counter for database keystone Oct 31 12:19:28.705927 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG oslo_db.sqlalchemy.engines [None req-612799c5-5f43-4c6e-9a8d-2ed8022a4354 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81492) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:326}} Oct 31 12:19:28.712015 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writer thread running {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 31 12:19:28.890395 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG passlib.handlers.bcrypt [None req-612799c5-5f43-4c6e-9a8d-2ed8022a4354 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81492) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 31 12:19:28.890699 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG passlib.handlers.bcrypt [None req-612799c5-5f43-4c6e-9a8d-2ed8022a4354 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81492) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 31 12:19:28.948487 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-612799c5-5f43-4c6e-9a8d-2ed8022a4354 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:19:28.950053 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-612799c5-5f43-4c6e-9a8d-2ed8022a4354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:19:29.077383 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 2/3] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:19:28 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 464 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 31 12:19:29.509121 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9daa74-0bf2-4c6d-90c2-3a4e91f45065 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:19:29.578603 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: INFO dbcounter [None req-cf9daa74-0bf2-4c6d-90c2-3a4e91f45065 None None] Registered counter for database keystone Oct 31 12:19:29.599722 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG oslo_db.sqlalchemy.engines [None req-cf9daa74-0bf2-4c6d-90c2-3a4e91f45065 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81491) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:326}} Oct 31 12:19:29.606338 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writer thread running {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 31 12:19:29.724771 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9daa74-0bf2-4c6d-90c2-3a4e91f45065 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:19:29.729524 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18d3a83-c357-4417-8a94-392a632a7a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:19:29.730139 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18d3a83-c357-4417-8a94-392a632a7a0f admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:29.730417 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18d3a83-c357-4417-8a94-392a632a7a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:29.730694 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18d3a83-c357-4417-8a94-392a632a7a0f admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:29.738815 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18d3a83-c357-4417-8a94-392a632a7a0f admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:19:29.790263 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: WARNING keystone.common.rbac_enforcer.enforcer [None req-b18d3a83-c357-4417-8a94-392a632a7a0f admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 31 12:19:29.790603 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18d3a83-c357-4417-8a94-392a632a7a0f admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:19:29.799896 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 2/4] 10.222.0.15 () {58 vars in 1324 bytes} [Mon Oct 31 12:19:29 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 31 12:19:29.805759 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612799c5-5f43-4c6e-9a8d-2ed8022a4354 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:19:29.823719 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612799c5-5f43-4c6e-9a8d-2ed8022a4354 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:19:29.828063 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57777109-759b-453e-a4e2-de5fc327edbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:19:29.829155 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57777109-759b-453e-a4e2-de5fc327edbf admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:29.829433 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57777109-759b-453e-a4e2-de5fc327edbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:29.829711 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57777109-759b-453e-a4e2-de5fc327edbf admin admin] PATH_INFO: `/v3/projects/20cfeec04f5e4394b2ab5ceaa7d9c63a` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:29.833796 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57777109-759b-453e-a4e2-de5fc327edbf admin admin] RBAC: Authorizing `identity:get_project(project_id=20cfeec04f5e4394b2ab5ceaa7d9c63a)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:19:29.883676 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: WARNING keystone.common.rbac_enforcer.enforcer [None req-57777109-759b-453e-a4e2-de5fc327edbf admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 31 12:19:29.884015 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57777109-759b-453e-a4e2-de5fc327edbf admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:19:29.889060 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 3/5] 10.222.0.15 () {56 vars in 1207 bytes} [Mon Oct 31 12:19:29 2022] GET /identity/v3/projects/20cfeec04f5e4394b2ab5ceaa7d9c63a => generated 344 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:19:31.984978 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5527989e-59b9-4d3a-87b6-0b754cb092e5 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:31.985516 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5527989e-59b9-4d3a-87b6-0b754cb092e5 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:31.985516 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5527989e-59b9-4d3a-87b6-0b754cb092e5 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:31.989620 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 3/6] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:19:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:19:31.995406 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2027ca21-f999-474a-841e-d376509d2821 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:31.995822 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2027ca21-f999-474a-841e-d376509d2821 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:31.995952 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2027ca21-f999-474a-841e-d376509d2821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:32.016643 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-2027ca21-f999-474a-841e-d376509d2821 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:19:32.018145 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-2027ca21-f999-474a-841e-d376509d2821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:19:32.026255 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 4/7] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:19:31 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 31 12:19:32.292681 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5527989e-59b9-4d3a-87b6-0b754cb092e5 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:19:32.304605 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5527989e-59b9-4d3a-87b6-0b754cb092e5 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:19:32.309289 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdd4ef9-a3a7-4d19-bfd8-d978b5e0b7b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:19:32.309887 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdd4ef9-a3a7-4d19-bfd8-d978b5e0b7b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:32.310165 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdd4ef9-a3a7-4d19-bfd8-d978b5e0b7b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:32.310435 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdd4ef9-a3a7-4d19-bfd8-d978b5e0b7b8 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:32.317939 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdd4ef9-a3a7-4d19-bfd8-d978b5e0b7b8 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:19:32.319739 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdd4ef9-a3a7-4d19-bfd8-d978b5e0b7b8 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:19:32.328899 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 4/8] 10.222.0.15 () {58 vars in 1324 bytes} [Mon Oct 31 12:19:32 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 31 12:19:32.334135 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2027ca21-f999-474a-841e-d376509d2821 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:19:32.343273 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2027ca21-f999-474a-841e-d376509d2821 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:19:32.347494 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d366b2d4-1fa7-428d-b775-076bc312bf74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:19:32.348104 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d366b2d4-1fa7-428d-b775-076bc312bf74 admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:32.348380 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d366b2d4-1fa7-428d-b775-076bc312bf74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:32.348650 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d366b2d4-1fa7-428d-b775-076bc312bf74 admin admin] PATH_INFO: `/v3/users/d2ddf15ba0c7488b8e49f974bba93de7` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:32.349924 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d366b2d4-1fa7-428d-b775-076bc312bf74 admin admin] RBAC: Authorizing `identity:get_user(user_id=d2ddf15ba0c7488b8e49f974bba93de7)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:19:32.351751 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d366b2d4-1fa7-428d-b775-076bc312bf74 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:19:32.353238 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 5/9] 10.222.0.15 () {56 vars in 1198 bytes} [Mon Oct 31 12:19:32 2022] GET /identity/v3/users/d2ddf15ba0c7488b8e49f974bba93de7 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:19:34.550229 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c749e21a-cb92-42f0-85d2-ddc56970443a None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:34.550750 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c749e21a-cb92-42f0-85d2-ddc56970443a None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:34.550750 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c749e21a-cb92-42f0-85d2-ddc56970443a None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:34.552120 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 5/10] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:19:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 31 12:19:34.557849 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16092ad4-50d0-4ffc-9306-ecb0c2b85657 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:34.558267 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16092ad4-50d0-4ffc-9306-ecb0c2b85657 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:34.558383 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16092ad4-50d0-4ffc-9306-ecb0c2b85657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:34.576605 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-16092ad4-50d0-4ffc-9306-ecb0c2b85657 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:19:34.577790 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-16092ad4-50d0-4ffc-9306-ecb0c2b85657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:19:34.594540 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439549da-303e-4f7f-a369-b3c29e2da0e0 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:34.594820 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439549da-303e-4f7f-a369-b3c29e2da0e0 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:34.595095 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439549da-303e-4f7f-a369-b3c29e2da0e0 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:34.596434 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 6/11] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:19:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 31 12:19:34.601475 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76acf543-fa79-4188-8f64-224534419352 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:34.601753 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76acf543-fa79-4188-8f64-224534419352 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:34.602022 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76acf543-fa79-4188-8f64-224534419352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:34.643990 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG passlib.handlers.bcrypt [None req-76acf543-fa79-4188-8f64-224534419352 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81491) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 31 12:19:34.644381 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG passlib.handlers.bcrypt [None req-76acf543-fa79-4188-8f64-224534419352 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81491) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 31 12:19:34.661033 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 6/12] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:19:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 31 12:19:34.693205 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-76acf543-fa79-4188-8f64-224534419352 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:19:34.694778 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-76acf543-fa79-4188-8f64-224534419352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:19:34.794018 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 7/13] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:19:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 31 12:19:34.929022 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16092ad4-50d0-4ffc-9306-ecb0c2b85657 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:19:34.953427 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:19:35.034760 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16092ad4-50d0-4ffc-9306-ecb0c2b85657 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:19:35.066749 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76acf543-fa79-4188-8f64-224534419352 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:19:35.068238 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4092b40-d577-47bb-a3bb-fd4f5cf23acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:19:35.068883 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4092b40-d577-47bb-a3bb-fd4f5cf23acc None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:35.069166 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4092b40-d577-47bb-a3bb-fd4f5cf23acc None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:35.069444 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4092b40-d577-47bb-a3bb-fd4f5cf23acc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:35.081627 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4092b40-d577-47bb-a3bb-fd4f5cf23acc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 31 12:19:35.090529 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-f4092b40-d577-47bb-a3bb-fd4f5cf23acc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 31 12:19:35.090529 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:19:35.090529 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:19:35.090529 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:19:35.090529 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:19:35.090529 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:19:35.090529 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:19:35.090529 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:19:35.090529 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:19:35.090529 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:19:35.090529 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:19:35.090529 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:19:35.090529 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 31 12:19:35.090529 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 31 12:19:35.090529 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 31 12:19:35.090529 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 31 12:19:35.091545 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 31 12:19:35.091545 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 31 12:19:35.091545 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 31 12:19:35.091545 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 31 12:19:35.091545 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:19:35.091545 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:19:35.091545 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:19:35.091545 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:19:35.091545 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:19:35.091545 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:19:35.091545 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:19:35.091545 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:19:35.091545 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:19:35.091545 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:19:35.091545 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:19:35.091545 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:19:35.092689 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:19:35.092689 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:19:35.092689 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:19:35.092689 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:19:35.092689 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 31 12:19:35.092689 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 31 12:19:35.092689 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 31 12:19:35.092689 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 31 12:19:35.092689 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 31 12:19:35.092689 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 31 12:19:35.092689 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 31 12:19:35.092689 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:19:35.092689 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 7/14] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Oct 31 12:19:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:19:35.097308 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4092b40-d577-47bb-a3bb-fd4f5cf23acc None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:19:35.168898 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76acf543-fa79-4188-8f64-224534419352 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:19:35.171222 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:19:35.179562 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:19:35.199387 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4092b40-d577-47bb-a3bb-fd4f5cf23acc None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:19:35.206441 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc62775d-cb11-43a5-a815-9a5461e29aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:19:35.207335 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc62775d-cb11-43a5-a815-9a5461e29aef None admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:35.207622 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc62775d-cb11-43a5-a815-9a5461e29aef None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:35.207895 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc62775d-cb11-43a5-a815-9a5461e29aef None admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:35.208863 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc62775d-cb11-43a5-a815-9a5461e29aef None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:19:35.210288 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc62775d-cb11-43a5-a815-9a5461e29aef None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:19:35.221967 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.sql.core [None req-cc62775d-cb11-43a5-a815-9a5461e29aef None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 31 12:19:35.221967 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 31 12:19:35.221967 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [parameters: {'id': '0d4d74263e20455bb7a0197e36dda234', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 31 12:19:35.221967 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81491) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 31 12:19:35.234175 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6888013-c575-45df-a7f2-43cc572f3400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:19:35.234808 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6888013-c575-45df-a7f2-43cc572f3400 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:35.235085 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6888013-c575-45df-a7f2-43cc572f3400 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:35.235387 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6888013-c575-45df-a7f2-43cc572f3400 None admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:35.236045 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6888013-c575-45df-a7f2-43cc572f3400 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:19:35.236342 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6888013-c575-45df-a7f2-43cc572f3400 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:19:35.237704 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6888013-c575-45df-a7f2-43cc572f3400 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:19:35.240778 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-cc62775d-cb11-43a5-a815-9a5461e29aef None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 31 12:19:35.240778 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:19:35.240778 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 31 12:19:35.240778 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 31 12:19:35.240778 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 31 12:19:35.240778 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 31 12:19:35.240778 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 31 12:19:35.240778 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application result = self._query(query) Oct 31 12:19:35.240778 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 31 12:19:35.240778 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application conn.query(q) Oct 31 12:19:35.240778 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 31 12:19:35.240778 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 31 12:19:35.240778 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 31 12:19:35.240778 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application result.read() Oct 31 12:19:35.241512 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 31 12:19:35.241512 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 31 12:19:35.241512 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 31 12:19:35.241512 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 31 12:19:35.241512 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 31 12:19:35.241512 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 31 12:19:35.241512 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 31 12:19:35.241512 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 31 12:19:35.241512 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 31 12:19:35.241512 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:19:35.241512 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 31 12:19:35.241512 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:19:35.241512 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:19:35.241512 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 31 12:19:35.241512 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 31 12:19:35.241512 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 31 12:19:35.241512 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 31 12:19:35.241512 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 31 12:19:35.242575 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application next(self.gen) Oct 31 12:19:35.242575 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Oct 31 12:19:35.242575 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application yield resource Oct 31 12:19:35.242575 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 31 12:19:35.242575 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application next(self.gen) Oct 31 12:19:35.242575 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Oct 31 12:19:35.242575 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self.session.rollback() Oct 31 12:19:35.242575 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 31 12:19:35.242575 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self.force_reraise() Oct 31 12:19:35.242575 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 31 12:19:35.242575 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise self.value Oct 31 12:19:35.242575 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Oct 31 12:19:35.242575 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 31 12:19:35.242575 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Oct 31 12:19:35.242575 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application session.commit() Oct 31 12:19:35.242575 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 31 12:19:35.242575 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 31 12:19:35.243601 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 31 12:19:35.243601 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self._prepare_impl() Oct 31 12:19:35.243601 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 31 12:19:35.243601 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self.session.flush() Oct 31 12:19:35.243601 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 31 12:19:35.243601 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self._flush(objects) Oct 31 12:19:35.243601 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 31 12:19:35.243601 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 31 12:19:35.243601 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 31 12:19:35.243601 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application compat.raise_( Oct 31 12:19:35.243601 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 31 12:19:35.243601 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception Oct 31 12:19:35.243601 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 31 12:19:35.243601 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application flush_context.execute() Oct 31 12:19:35.243601 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 31 12:19:35.243601 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rec.execute(self) Oct 31 12:19:35.243601 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 31 12:19:35.244615 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 31 12:19:35.244615 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 31 12:19:35.244615 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 31 12:19:35.244615 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 31 12:19:35.244615 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 31 12:19:35.244615 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 31 12:19:35.244615 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 31 12:19:35.244615 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 31 12:19:35.244615 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 31 12:19:35.244615 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 31 12:19:35.244615 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 31 12:19:35.244615 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 31 12:19:35.244615 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 31 12:19:35.244615 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 31 12:19:35.244615 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 31 12:19:35.245372 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 31 12:19:35.245372 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception Oct 31 12:19:35.245372 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 31 12:19:35.245372 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 31 12:19:35.245372 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 31 12:19:35.245372 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 31 12:19:35.245372 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 31 12:19:35.245372 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application result = self._query(query) Oct 31 12:19:35.245372 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 31 12:19:35.245372 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application conn.query(q) Oct 31 12:19:35.245372 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 31 12:19:35.245372 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 31 12:19:35.245372 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 31 12:19:35.245372 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application result.read() Oct 31 12:19:35.245372 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 31 12:19:35.245372 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 31 12:19:35.245372 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 31 12:19:35.246571 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 31 12:19:35.246571 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 31 12:19:35.246571 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 31 12:19:35.246571 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 31 12:19:35.246571 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 31 12:19:35.246571 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 31 12:19:35.246571 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 31 12:19:35.246571 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [parameters: {'id': '0d4d74263e20455bb7a0197e36dda234', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 31 12:19:35.246571 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 31 12:19:35.246571 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:19:35.246571 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 31 12:19:35.246571 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:19:35.246571 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:19:35.246571 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 31 12:19:35.246571 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 31 12:19:35.247447 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 31 12:19:35.247447 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 31 12:19:35.247447 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 31 12:19:35.247447 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:19:35.247447 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 31 12:19:35.247447 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:19:35.247447 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:19:35.247447 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:19:35.247447 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:19:35.247447 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:19:35.247447 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:19:35.247447 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:19:35.247447 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:19:35.247447 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:19:35.247447 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:19:35.247447 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:19:35.247447 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:19:35.248632 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 31 12:19:35.248632 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 31 12:19:35.248632 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:19:35.248632 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:19:35.248632 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 31 12:19:35.248632 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 31 12:19:35.248632 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:19:35.248632 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:19:35.248632 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 31 12:19:35.248632 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 31 12:19:35.248632 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 31 12:19:35.248632 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:19:35.248632 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 8/15] 10.222.0.15 () {60 vars in 1137 bytes} [Mon Oct 31 12:19:35 2022] POST /identity/v3/domains => generated 190 bytes in 176 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 31 12:19:35.248632 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc62775d-cb11-43a5-a815-9a5461e29aef None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:19:35.252514 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 8/16] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Oct 31 12:19:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 31 12:19:35.258618 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6888013-c575-45df-a7f2-43cc572f3400 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:19:35.348566 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc62775d-cb11-43a5-a815-9a5461e29aef None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:19:35.359519 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6888013-c575-45df-a7f2-43cc572f3400 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:19:35.380722 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7eb99f-3261-475a-b03c-c93f025dbf5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:19:35.381569 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7eb99f-3261-475a-b03c-c93f025dbf5e None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:35.381850 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7eb99f-3261-475a-b03c-c93f025dbf5e None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:35.382116 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7eb99f-3261-475a-b03c-c93f025dbf5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:35.393637 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f88c36-fe3a-4850-8cf9-b7c8daa4d0ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:19:35.394265 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f88c36-fe3a-4850-8cf9-b7c8daa4d0ed None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:35.394545 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f88c36-fe3a-4850-8cf9-b7c8daa4d0ed None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:35.394819 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f88c36-fe3a-4850-8cf9-b7c8daa4d0ed None admin] PATH_INFO: `/v3/users/d2ddf15ba0c7488b8e49f974bba93de7` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:35.395991 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f88c36-fe3a-4850-8cf9-b7c8daa4d0ed None admin] RBAC: Authorizing `identity:get_user(user_id=d2ddf15ba0c7488b8e49f974bba93de7)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:19:35.397432 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f88c36-fe3a-4850-8cf9-b7c8daa4d0ed None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:19:35.398999 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 9/17] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Oct 31 12:19:35 2022] GET /identity/v3/users/d2ddf15ba0c7488b8e49f974bba93de7 => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 31 12:19:35.400309 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7eb99f-3261-475a-b03c-c93f025dbf5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:19:35.401842 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7eb99f-3261-475a-b03c-c93f025dbf5e None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:19:35.404440 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f88c36-fe3a-4850-8cf9-b7c8daa4d0ed None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:19:35.418004 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-1c7eb99f-3261-475a-b03c-c93f025dbf5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:19:35.418004 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:19:35.418004 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:19:35.418004 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:19:35.418004 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:19:35.418004 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:19:35.418004 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:19:35.418004 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:19:35.418004 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:19:35.418004 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:19:35.418004 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:19:35.418004 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:19:35.418004 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 31 12:19:35.418004 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 31 12:19:35.418004 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 31 12:19:35.418004 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 31 12:19:35.418982 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:19:35.418982 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:19:35.418982 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:19:35.418982 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:19:35.418982 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:19:35.418982 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:19:35.418982 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:19:35.418982 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 31 12:19:35.418982 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:19:35.418982 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:19:35.418982 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:19:35.418982 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:19:35.418982 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:19:35.418982 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:19:35.418982 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:19:35.418982 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:19:35.418982 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 31 12:19:35.420067 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 31 12:19:35.420067 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:19:35.420067 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:19:35.420233 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:19:35.420305 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 9/18] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Oct 31 12:19:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 31 12:19:35.424777 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7eb99f-3261-475a-b03c-c93f025dbf5e None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:19:35.449838 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:19:35.505727 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f88c36-fe3a-4850-8cf9-b7c8daa4d0ed None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:19:35.527883 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7eb99f-3261-475a-b03c-c93f025dbf5e None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:19:35.538732 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394b971b-6f05-4a49-aee1-a25828fbd83d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:19:35.539384 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394b971b-6f05-4a49-aee1-a25828fbd83d None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:35.539668 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394b971b-6f05-4a49-aee1-a25828fbd83d None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:35.539935 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394b971b-6f05-4a49-aee1-a25828fbd83d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:35.541534 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394b971b-6f05-4a49-aee1-a25828fbd83d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:19:35.542967 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394b971b-6f05-4a49-aee1-a25828fbd83d None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:19:35.544532 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 10/19] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Oct 31 12:19:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 31 12:19:35.550001 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394b971b-6f05-4a49-aee1-a25828fbd83d None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:19:35.564091 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbeb846-26e6-4dae-8ef1-7bbde583310a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:19:35.564696 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbeb846-26e6-4dae-8ef1-7bbde583310a None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:35.564971 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbeb846-26e6-4dae-8ef1-7bbde583310a None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:35.565240 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbeb846-26e6-4dae-8ef1-7bbde583310a None admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:35.565868 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbeb846-26e6-4dae-8ef1-7bbde583310a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:19:35.566164 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbeb846-26e6-4dae-8ef1-7bbde583310a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:19:35.567545 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbeb846-26e6-4dae-8ef1-7bbde583310a None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:19:35.584834 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 10/20] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Oct 31 12:19:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 31 12:19:35.639933 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:19:35.661251 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394b971b-6f05-4a49-aee1-a25828fbd83d None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:19:35.696252 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11f8d0d-98cd-474b-84c6-e662b8db38c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:19:35.696929 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11f8d0d-98cd-474b-84c6-e662b8db38c7 None admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:35.697763 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11f8d0d-98cd-474b-84c6-e662b8db38c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:35.698190 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11f8d0d-98cd-474b-84c6-e662b8db38c7 None admin] PATH_INFO: `/v3/domains/default/users/d2ddf15ba0c7488b8e49f974bba93de7/roles/176c07b4fe3b4537bdfeb4be21b379c5` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:35.700292 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11f8d0d-98cd-474b-84c6-e662b8db38c7 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=d2ddf15ba0c7488b8e49f974bba93de7, role_id=176c07b4fe3b4537bdfeb4be21b379c5)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:19:35.701685 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11f8d0d-98cd-474b-84c6-e662b8db38c7 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:19:35.816696 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 11/21] 10.222.0.15 () {58 vars in 1361 bytes} [Mon Oct 31 12:19:35 2022] PUT /identity/v3/domains/default/users/d2ddf15ba0c7488b8e49f974bba93de7/roles/176c07b4fe3b4537bdfeb4be21b379c5 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 31 12:19:38.324383 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31012936-eb15-408e-8dc2-ec7ca5f2a202 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:38.324860 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31012936-eb15-408e-8dc2-ec7ca5f2a202 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:38.324860 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31012936-eb15-408e-8dc2-ec7ca5f2a202 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:38.326166 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 11/22] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:19:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:19:38.342474 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a633f674-6ff7-45aa-ad56-59e641667f4a None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:38.343416 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a633f674-6ff7-45aa-ad56-59e641667f4a None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:38.343886 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a633f674-6ff7-45aa-ad56-59e641667f4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:38.368557 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-a633f674-6ff7-45aa-ad56-59e641667f4a None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:19:38.370502 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-a633f674-6ff7-45aa-ad56-59e641667f4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:19:38.398888 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07380c20-7ccc-4dae-b1f9-aa837b984d11 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:38.398888 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07380c20-7ccc-4dae-b1f9-aa837b984d11 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:38.398888 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07380c20-7ccc-4dae-b1f9-aa837b984d11 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:38.398888 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 12/23] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:19:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 31 12:19:38.411030 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c018028-2d7a-431d-b710-208d1305f59b None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:38.411764 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c018028-2d7a-431d-b710-208d1305f59b None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:38.412251 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c018028-2d7a-431d-b710-208d1305f59b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:38.446464 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-2c018028-2d7a-431d-b710-208d1305f59b None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:19:38.448572 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-2c018028-2d7a-431d-b710-208d1305f59b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:19:38.488476 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 12/24] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:19:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 31 12:19:38.561793 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 13/25] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:19:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 31 12:19:38.816492 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7255d0-2871-4504-9d23-86416cec6692 None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:38.816857 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7255d0-2871-4504-9d23-86416cec6692 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:38.817042 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7255d0-2871-4504-9d23-86416cec6692 None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:38.821324 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 13/26] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:19:38 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:19:38.832701 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a28b57-9b95-4612-86ef-0f0b8d2916ac None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:38.832990 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a28b57-9b95-4612-86ef-0f0b8d2916ac None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:38.833258 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a28b57-9b95-4612-86ef-0f0b8d2916ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:38.845982 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0834da-5a6d-49d4-a3f8-07bf2f027290 None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:38.846267 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0834da-5a6d-49d4-a3f8-07bf2f027290 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:38.846528 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0834da-5a6d-49d4-a3f8-07bf2f027290 None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:38.847907 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 14/27] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:19:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 31 12:19:38.859220 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-b8a28b57-9b95-4612-86ef-0f0b8d2916ac None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:19:38.860395 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-b8a28b57-9b95-4612-86ef-0f0b8d2916ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:19:38.864279 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:19:38.873045 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd900e10-6e7a-442c-a652-436afb585f90 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:38.873327 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd900e10-6e7a-442c-a652-436afb585f90 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:38.873598 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd900e10-6e7a-442c-a652-436afb585f90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:38.904862 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-dd900e10-6e7a-442c-a652-436afb585f90 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:19:38.906051 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-dd900e10-6e7a-442c-a652-436afb585f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:19:38.988948 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 14/28] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:19:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 31 12:19:38.994730 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a28b57-9b95-4612-86ef-0f0b8d2916ac None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:19:39.048496 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:19:39.068856 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 15/29] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:19:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 31 12:19:39.069377 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd900e10-6e7a-442c-a652-436afb585f90 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:19:39.139938 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a28b57-9b95-4612-86ef-0f0b8d2916ac None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:19:39.207436 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d798d36-c5ee-4335-88c9-27c36bfdcdc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:19:39.208513 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d798d36-c5ee-4335-88c9-27c36bfdcdc4 None admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:39.209004 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d798d36-c5ee-4335-88c9-27c36bfdcdc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:39.209466 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d798d36-c5ee-4335-88c9-27c36bfdcdc4 None admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:39.210315 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d798d36-c5ee-4335-88c9-27c36bfdcdc4 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:19:39.212048 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d798d36-c5ee-4335-88c9-27c36bfdcdc4 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:19:39.240565 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:19:39.306796 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd900e10-6e7a-442c-a652-436afb585f90 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:19:39.364674 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 15/30] 10.222.0.15 () {60 vars in 1131 bytes} [Mon Oct 31 12:19:38 2022] POST /identity/v3/roles => generated 219 bytes in 375 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:19:39.366545 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d798d36-c5ee-4335-88c9-27c36bfdcdc4 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:19:39.390154 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd852695-7898-4ba5-9bf0-1416c306cf6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:19:39.393260 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd852695-7898-4ba5-9bf0-1416c306cf6f None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:39.393741 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd852695-7898-4ba5-9bf0-1416c306cf6f None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:39.394207 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd852695-7898-4ba5-9bf0-1416c306cf6f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:39.417861 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd852695-7898-4ba5-9bf0-1416c306cf6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 31 12:19:39.426206 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-dd852695-7898-4ba5-9bf0-1416c306cf6f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 31 12:19:39.426206 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:19:39.426206 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:19:39.426206 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:19:39.426206 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:19:39.426206 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:19:39.426206 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:19:39.426206 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:19:39.426206 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:19:39.426206 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:19:39.426206 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:19:39.426206 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:19:39.426206 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 31 12:19:39.426206 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 31 12:19:39.426206 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 31 12:19:39.426206 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 31 12:19:39.427128 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 31 12:19:39.427128 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 31 12:19:39.427128 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 31 12:19:39.427128 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 31 12:19:39.427128 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:19:39.427128 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:19:39.427128 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:19:39.427128 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:19:39.427128 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:19:39.427128 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:19:39.427128 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:19:39.427128 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:19:39.427128 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:19:39.427128 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:19:39.427128 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:19:39.427128 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:19:39.428009 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:19:39.428009 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:19:39.428009 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:19:39.428009 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:19:39.428009 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 31 12:19:39.428009 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 31 12:19:39.428009 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 31 12:19:39.428009 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 31 12:19:39.428009 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 31 12:19:39.428009 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 31 12:19:39.428009 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 31 12:19:39.428009 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:19:39.433558 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 16/31] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Oct 31 12:19:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 364 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:19:39.443132 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd852695-7898-4ba5-9bf0-1416c306cf6f None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:19:39.519642 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:19:39.542738 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d798d36-c5ee-4335-88c9-27c36bfdcdc4 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:19:39.582768 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd852695-7898-4ba5-9bf0-1416c306cf6f None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:19:39.587562 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f49f3f-91e5-43d5-a266-066cc3204289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:19:39.589255 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f49f3f-91e5-43d5-a266-066cc3204289 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:39.589545 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f49f3f-91e5-43d5-a266-066cc3204289 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:39.589818 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f49f3f-91e5-43d5-a266-066cc3204289 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:39.606949 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f49f3f-91e5-43d5-a266-066cc3204289 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:19:39.609096 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f49f3f-91e5-43d5-a266-066cc3204289 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:19:39.630123 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-d2f49f3f-91e5-43d5-a266-066cc3204289 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:19:39.630123 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:19:39.630123 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:19:39.630123 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:19:39.630123 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:19:39.630123 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:19:39.630123 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:19:39.630123 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:19:39.630123 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:19:39.630123 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:19:39.630123 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:19:39.630123 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:19:39.630123 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 31 12:19:39.630123 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 31 12:19:39.630123 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 31 12:19:39.630123 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 31 12:19:39.630979 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:19:39.630979 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:19:39.630979 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:19:39.630979 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:19:39.630979 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:19:39.630979 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:19:39.630979 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:19:39.630979 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 31 12:19:39.630979 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:19:39.630979 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:19:39.630979 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:19:39.630979 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:19:39.630979 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:19:39.630979 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:19:39.630979 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:19:39.630979 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:19:39.630979 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 31 12:19:39.631829 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 31 12:19:39.631829 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:19:39.631829 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:19:39.631985 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a534c490-08c7-4d2d-92e7-d3146a31fa6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:19:39.631985 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a534c490-08c7-4d2d-92e7-d3146a31fa6a None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:39.631985 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a534c490-08c7-4d2d-92e7-d3146a31fa6a None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:39.631985 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a534c490-08c7-4d2d-92e7-d3146a31fa6a None admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:39.635906 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a534c490-08c7-4d2d-92e7-d3146a31fa6a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:19:39.635906 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a534c490-08c7-4d2d-92e7-d3146a31fa6a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:19:39.635906 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a534c490-08c7-4d2d-92e7-d3146a31fa6a None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:19:39.636071 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 16/32] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Oct 31 12:19:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:19:39.636071 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f49f3f-91e5-43d5-a266-066cc3204289 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:19:39.644674 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 17/33] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Oct 31 12:19:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:19:39.646888 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18903414-46a4-44e4-9f2b-8e6e870346c0 None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:39.648190 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18903414-46a4-44e4-9f2b-8e6e870346c0 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:39.648190 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18903414-46a4-44e4-9f2b-8e6e870346c0 None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:39.648894 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 18/34] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:19:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:19:39.650279 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18903414-46a4-44e4-9f2b-8e6e870346c0 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:19:39.711970 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:19:39.764986 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f49f3f-91e5-43d5-a266-066cc3204289 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:19:39.771868 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18903414-46a4-44e4-9f2b-8e6e870346c0 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:19:39.799085 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:19:39.802898 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ccbfb8-0669-44d5-8f1f-4c509f9fdf6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:19:39.803562 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ccbfb8-0669-44d5-8f1f-4c509f9fdf6f None admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:39.803847 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ccbfb8-0669-44d5-8f1f-4c509f9fdf6f None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:39.804123 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ccbfb8-0669-44d5-8f1f-4c509f9fdf6f None admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:39.804798 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ccbfb8-0669-44d5-8f1f-4c509f9fdf6f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:19:39.806184 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ccbfb8-0669-44d5-8f1f-4c509f9fdf6f None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:19:39.813534 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d38961-e4d7-4793-9ec6-6db030243f2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:19:39.814223 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d38961-e4d7-4793-9ec6-6db030243f2b None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:39.814506 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d38961-e4d7-4793-9ec6-6db030243f2b None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:39.814814 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d38961-e4d7-4793-9ec6-6db030243f2b None admin] PATH_INFO: `/v3/domains` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:39.815508 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d38961-e4d7-4793-9ec6-6db030243f2b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:19:39.815834 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d38961-e4d7-4793-9ec6-6db030243f2b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:19:39.817270 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d38961-e4d7-4793-9ec6-6db030243f2b None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:19:39.821899 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 17/35] 10.222.0.15 () {60 vars in 1131 bytes} [Mon Oct 31 12:19:39 2022] POST /identity/v3/roles => generated 225 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:19:39.823386 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ccbfb8-0669-44d5-8f1f-4c509f9fdf6f None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:19:39.843543 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 19/36] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Oct 31 12:19:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:19:39.843543 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdd3d55-0543-4d6e-8328-5123ba173e62 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:39.843543 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdd3d55-0543-4d6e-8328-5123ba173e62 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:39.843543 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdd3d55-0543-4d6e-8328-5123ba173e62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:39.861370 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-cfdd3d55-0543-4d6e-8328-5123ba173e62 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:19:39.862630 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-cfdd3d55-0543-4d6e-8328-5123ba173e62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:19:39.958185 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 20/37] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:19:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 31 12:19:39.959720 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdd3d55-0543-4d6e-8328-5123ba173e62 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:19:39.964721 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ccbfb8-0669-44d5-8f1f-4c509f9fdf6f None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:19:39.980962 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:19:40.012329 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab98f4d-ca0b-4fb3-a2ad-b3fd32a26b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:19:40.012992 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab98f4d-ca0b-4fb3-a2ad-b3fd32a26b44 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:40.013275 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab98f4d-ca0b-4fb3-a2ad-b3fd32a26b44 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:40.013550 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab98f4d-ca0b-4fb3-a2ad-b3fd32a26b44 None admin] PATH_INFO: `/v3/users/d2ddf15ba0c7488b8e49f974bba93de7` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:40.014984 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab98f4d-ca0b-4fb3-a2ad-b3fd32a26b44 None admin] RBAC: Authorizing `identity:get_user(user_id=d2ddf15ba0c7488b8e49f974bba93de7)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:19:40.016845 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab98f4d-ca0b-4fb3-a2ad-b3fd32a26b44 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:19:40.018457 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 18/38] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Oct 31 12:19:39 2022] GET /identity/v3/users/d2ddf15ba0c7488b8e49f974bba93de7 => generated 247 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:19:40.026210 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab98f4d-ca0b-4fb3-a2ad-b3fd32a26b44 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:19:40.116696 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdd3d55-0543-4d6e-8328-5123ba173e62 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:19:40.134054 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:19:40.146106 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab98f4d-ca0b-4fb3-a2ad-b3fd32a26b44 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:19:40.160918 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976a3d51-cf76-4d38-9443-3f7526f1a6aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:19:40.161698 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976a3d51-cf76-4d38-9443-3f7526f1a6aa None admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:40.162163 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976a3d51-cf76-4d38-9443-3f7526f1a6aa None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:40.162608 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976a3d51-cf76-4d38-9443-3f7526f1a6aa None admin] PATH_INFO: `/v3/projects` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:40.163830 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976a3d51-cf76-4d38-9443-3f7526f1a6aa None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:19:40.165936 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976a3d51-cf76-4d38-9443-3f7526f1a6aa None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:19:40.186531 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f30844-f115-4508-9770-b5b7ed924be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:19:40.187196 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f30844-f115-4508-9770-b5b7ed924be1 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:40.188374 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f30844-f115-4508-9770-b5b7ed924be1 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:40.188594 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f30844-f115-4508-9770-b5b7ed924be1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:40.190264 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f30844-f115-4508-9770-b5b7ed924be1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:19:40.191813 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f30844-f115-4508-9770-b5b7ed924be1 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:19:40.194091 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 19/39] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Oct 31 12:19:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 31 12:19:40.195306 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39acdf9-be5a-487c-9508-e10441a37057 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:40.195597 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39acdf9-be5a-487c-9508-e10441a37057 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:40.195878 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39acdf9-be5a-487c-9508-e10441a37057 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:40.197188 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 20/40] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:19:40 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:19:40.202049 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39acdf9-be5a-487c-9508-e10441a37057 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:19:40.205280 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 21/41] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Oct 31 12:19:39 2022] POST /identity/v3/projects => generated 301 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:19:40.207731 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b356c6d6-502a-4596-9e9f-0789b0abe4c2 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:40.208015 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b356c6d6-502a-4596-9e9f-0789b0abe4c2 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:40.208350 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b356c6d6-502a-4596-9e9f-0789b0abe4c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:40.270576 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:19:40.343374 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39acdf9-be5a-487c-9508-e10441a37057 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:19:40.351574 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-b356c6d6-502a-4596-9e9f-0789b0abe4c2 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:19:40.352788 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-b356c6d6-502a-4596-9e9f-0789b0abe4c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:19:40.379285 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8fb830-a2ac-4b5e-a674-14c946117b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:19:40.379921 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8fb830-a2ac-4b5e-a674-14c946117b53 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:40.380200 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8fb830-a2ac-4b5e-a674-14c946117b53 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:40.380479 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8fb830-a2ac-4b5e-a674-14c946117b53 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:40.381210 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8fb830-a2ac-4b5e-a674-14c946117b53 None admin] RBAC: Adding query filter params (role.id=176c07b4fe3b4537bdfeb4be21b379c5, scope.domain.id=default, user.id=d2ddf15ba0c7488b8e49f974bba93de7) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:19:40.381516 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8fb830-a2ac-4b5e-a674-14c946117b53 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:19:40.383203 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8fb830-a2ac-4b5e-a674-14c946117b53 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:19:40.393461 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 21/42] 10.222.0.15 () {56 vars in 1322 bytes} [Mon Oct 31 12:19:40 2022] GET /identity/v3/role_assignments?user.id=d2ddf15ba0c7488b8e49f974bba93de7&scope.domain.id=default&role.id=176c07b4fe3b4537bdfeb4be21b379c5 => generated 530 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:19:40.394300 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8fb830-a2ac-4b5e-a674-14c946117b53 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:19:40.460920 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 22/43] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:19:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 31 12:19:40.527708 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8fb830-a2ac-4b5e-a674-14c946117b53 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:19:40.542778 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:19:40.568545 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec3675e-f156-4f3e-a877-cf758f2ee918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:19:40.569169 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec3675e-f156-4f3e-a877-cf758f2ee918 None admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:40.569447 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec3675e-f156-4f3e-a877-cf758f2ee918 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:40.569719 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec3675e-f156-4f3e-a877-cf758f2ee918 None admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:40.570397 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec3675e-f156-4f3e-a877-cf758f2ee918 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:19:40.571802 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec3675e-f156-4f3e-a877-cf758f2ee918 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:19:40.590313 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 22/44] 10.222.0.15 () {60 vars in 1131 bytes} [Mon Oct 31 12:19:40 2022] POST /identity/v3/roles => generated 223 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:19:40.749931 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b356c6d6-502a-4596-9e9f-0789b0abe4c2 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:19:40.850190 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:19:40.858588 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b356c6d6-502a-4596-9e9f-0789b0abe4c2 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:19:40.891432 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70304b5b-2bb5-44c3-873e-5613a02c85e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:19:40.892310 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70304b5b-2bb5-44c3-873e-5613a02c85e8 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:40.892588 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70304b5b-2bb5-44c3-873e-5613a02c85e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:40.892854 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70304b5b-2bb5-44c3-873e-5613a02c85e8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:40.893931 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70304b5b-2bb5-44c3-873e-5613a02c85e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:19:40.895381 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70304b5b-2bb5-44c3-873e-5613a02c85e8 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:19:40.896925 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 23/45] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Oct 31 12:19:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:19:40.903211 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec3675e-f156-4f3e-a877-cf758f2ee918 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:19:40.999252 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec3675e-f156-4f3e-a877-cf758f2ee918 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:19:41.032731 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f180ff-49f6-4645-8f4a-da709f818179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:19:41.033342 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f180ff-49f6-4645-8f4a-da709f818179 None admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:41.033623 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f180ff-49f6-4645-8f4a-da709f818179 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:41.033899 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f180ff-49f6-4645-8f4a-da709f818179 None admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:41.034544 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f180ff-49f6-4645-8f4a-da709f818179 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:19:41.036008 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f180ff-49f6-4645-8f4a-da709f818179 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:19:41.062468 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 23/46] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Oct 31 12:19:40 2022] POST /identity/v3/projects => generated 312 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:19:43.212806 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3ca654-feba-4950-8215-71609555f7d3 None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:43.213365 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3ca654-feba-4950-8215-71609555f7d3 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:43.213513 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3ca654-feba-4950-8215-71609555f7d3 None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:43.214904 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 24/47] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:19:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:19:43.220023 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bfd22d-2695-4e05-90cc-bb2a8a520288 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:43.220451 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bfd22d-2695-4e05-90cc-bb2a8a520288 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:43.220584 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bfd22d-2695-4e05-90cc-bb2a8a520288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:43.238538 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-e8bfd22d-2695-4e05-90cc-bb2a8a520288 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:19:43.240057 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-e8bfd22d-2695-4e05-90cc-bb2a8a520288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:19:43.257473 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:19:43.306438 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 24/48] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:19:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 31 12:19:43.569682 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3ca654-feba-4950-8215-71609555f7d3 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:19:43.669373 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3ca654-feba-4950-8215-71609555f7d3 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:19:43.704245 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5934f3-b486-43ee-85fd-c899ab9a1703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:19:43.704863 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5934f3-b486-43ee-85fd-c899ab9a1703 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:43.705144 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5934f3-b486-43ee-85fd-c899ab9a1703 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:43.705415 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5934f3-b486-43ee-85fd-c899ab9a1703 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:43.706504 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5934f3-b486-43ee-85fd-c899ab9a1703 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:19:43.707967 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5934f3-b486-43ee-85fd-c899ab9a1703 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:19:43.709521 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 25/49] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Oct 31 12:19:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:19:43.715181 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bfd22d-2695-4e05-90cc-bb2a8a520288 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:19:43.809778 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bfd22d-2695-4e05-90cc-bb2a8a520288 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:19:43.842908 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19e065f-e096-4dd4-9711-aa73868efa15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:19:43.843472 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19e065f-e096-4dd4-9711-aa73868efa15 None admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:43.843750 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19e065f-e096-4dd4-9711-aa73868efa15 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:43.844024 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19e065f-e096-4dd4-9711-aa73868efa15 None admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:43.844683 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19e065f-e096-4dd4-9711-aa73868efa15 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:19:43.846103 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19e065f-e096-4dd4-9711-aa73868efa15 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:19:43.870446 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 25/50] 10.222.0.15 () {60 vars in 1140 bytes} [Mon Oct 31 12:19:43 2022] POST /identity/v3/projects => generated 298 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:19:45.985296 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8abea42-c52c-4c26-8c5f-5be874ffbdbb None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:45.985814 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8abea42-c52c-4c26-8c5f-5be874ffbdbb None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:45.985913 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8abea42-c52c-4c26-8c5f-5be874ffbdbb None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:45.987231 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 26/51] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:19:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:19:45.992454 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9a2b71-1334-41df-a8f9-5ee85afaf7d5 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:45.992857 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9a2b71-1334-41df-a8f9-5ee85afaf7d5 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:45.992997 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9a2b71-1334-41df-a8f9-5ee85afaf7d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:46.006347 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:19:46.012355 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-5e9a2b71-1334-41df-a8f9-5ee85afaf7d5 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:19:46.013664 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-5e9a2b71-1334-41df-a8f9-5ee85afaf7d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:19:46.079771 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 26/52] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:19:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 31 12:19:46.358677 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8abea42-c52c-4c26-8c5f-5be874ffbdbb None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:19:46.389804 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:19:46.458972 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8abea42-c52c-4c26-8c5f-5be874ffbdbb None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:19:46.491182 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222c7352-2241-4c35-8485-f6d353f3fce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:19:46.491817 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222c7352-2241-4c35-8485-f6d353f3fce2 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:46.492097 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222c7352-2241-4c35-8485-f6d353f3fce2 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:46.492368 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222c7352-2241-4c35-8485-f6d353f3fce2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:46.493461 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222c7352-2241-4c35-8485-f6d353f3fce2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:19:46.495706 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222c7352-2241-4c35-8485-f6d353f3fce2 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:19:46.497258 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 27/53] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Oct 31 12:19:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:19:46.502758 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9a2b71-1334-41df-a8f9-5ee85afaf7d5 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:19:46.599778 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9a2b71-1334-41df-a8f9-5ee85afaf7d5 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:19:46.737332 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3a76d3-2750-4a41-a351-656bd42c0de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:19:46.737850 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3a76d3-2750-4a41-a351-656bd42c0de7 None admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:46.738130 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3a76d3-2750-4a41-a351-656bd42c0de7 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:46.738403 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3a76d3-2750-4a41-a351-656bd42c0de7 None admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:46.739037 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3a76d3-2750-4a41-a351-656bd42c0de7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:19:46.740470 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3a76d3-2750-4a41-a351-656bd42c0de7 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:19:46.753179 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:19:46.768253 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 27/54] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Oct 31 12:19:46 2022] POST /identity/v3/users => generated 275 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:19:50.042762 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cbaafe-024d-4940-8846-0b6bf31b9581 None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:50.043284 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cbaafe-024d-4940-8846-0b6bf31b9581 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:50.043379 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cbaafe-024d-4940-8846-0b6bf31b9581 None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:50.045026 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 28/55] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:19:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:19:50.049905 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2912dec5-f291-4240-a335-c40caacca9c9 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:50.050305 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2912dec5-f291-4240-a335-c40caacca9c9 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:50.050459 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2912dec5-f291-4240-a335-c40caacca9c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:50.071923 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-2912dec5-f291-4240-a335-c40caacca9c9 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:19:50.073996 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-2912dec5-f291-4240-a335-c40caacca9c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:19:50.146560 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 28/56] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:19:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 31 12:19:50.150414 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbd1ba9-4f65-4993-bfc4-65df1e6c9ec2 None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:50.150908 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbd1ba9-4f65-4993-bfc4-65df1e6c9ec2 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:50.151363 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbd1ba9-4f65-4993-bfc4-65df1e6c9ec2 None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:50.152919 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 29/57] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:19:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:19:50.158206 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f93defa-076a-49f3-b144-89e748f49477 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:50.158650 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f93defa-076a-49f3-b144-89e748f49477 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:50.159095 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f93defa-076a-49f3-b144-89e748f49477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:50.184267 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-3f93defa-076a-49f3-b144-89e748f49477 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:19:50.185687 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-3f93defa-076a-49f3-b144-89e748f49477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:19:50.280520 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 29/58] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:19:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 31 12:19:50.397573 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125be9e1-1a10-4815-906c-6926ab2d749f None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:50.398224 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125be9e1-1a10-4815-906c-6926ab2d749f None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:50.398696 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125be9e1-1a10-4815-906c-6926ab2d749f None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:50.400300 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 30/59] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:19:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:19:50.405743 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48256f53-08db-4bea-a9f5-b011b56f71bb None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:50.406039 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48256f53-08db-4bea-a9f5-b011b56f71bb None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:50.406321 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48256f53-08db-4bea-a9f5-b011b56f71bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:50.436236 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-48256f53-08db-4bea-a9f5-b011b56f71bb None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:19:50.437437 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-48256f53-08db-4bea-a9f5-b011b56f71bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:19:50.500816 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:19:50.528542 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 30/60] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:19:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 31 12:19:50.617857 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125be9e1-1a10-4815-906c-6926ab2d749f None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:19:50.627765 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48256f53-08db-4bea-a9f5-b011b56f71bb None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:19:50.744030 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:19:50.750867 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125be9e1-1a10-4815-906c-6926ab2d749f None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:19:50.752042 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48256f53-08db-4bea-a9f5-b011b56f71bb None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:19:50.786212 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5e2366-3a90-471a-8ec5-4ed526c8fbab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:19:50.786874 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5e2366-3a90-471a-8ec5-4ed526c8fbab None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:50.787154 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5e2366-3a90-471a-8ec5-4ed526c8fbab None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:50.787449 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5e2366-3a90-471a-8ec5-4ed526c8fbab None admin] PATH_INFO: `/v3/roles/member` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:50.789000 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2f9a56-b62c-48cb-822a-1224ad1cddd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:19:50.789639 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2f9a56-b62c-48cb-822a-1224ad1cddd8 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:50.789917 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2f9a56-b62c-48cb-822a-1224ad1cddd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:50.790191 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2f9a56-b62c-48cb-822a-1224ad1cddd8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:50.798031 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5e2366-3a90-471a-8ec5-4ed526c8fbab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 31 12:19:50.802017 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2f9a56-b62c-48cb-822a-1224ad1cddd8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 31 12:19:50.803196 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-af5e2366-3a90-471a-8ec5-4ed526c8fbab None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 31 12:19:50.803196 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:19:50.803196 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:19:50.803196 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:19:50.803196 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:19:50.803196 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:19:50.803196 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:19:50.803196 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:19:50.803196 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:19:50.803196 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:19:50.803196 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:19:50.803196 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:19:50.803196 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 31 12:19:50.803196 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 31 12:19:50.803196 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 31 12:19:50.803196 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 31 12:19:50.803993 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 31 12:19:50.803993 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 31 12:19:50.803993 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 31 12:19:50.803993 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 31 12:19:50.803993 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:19:50.803993 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:19:50.803993 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:19:50.803993 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:19:50.803993 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:19:50.803993 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:19:50.803993 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:19:50.803993 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:19:50.803993 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:19:50.803993 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:19:50.803993 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:19:50.803993 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:19:50.804749 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:19:50.804749 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:19:50.804749 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:19:50.804749 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:19:50.804749 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 31 12:19:50.804749 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 31 12:19:50.804749 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 31 12:19:50.804749 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 31 12:19:50.804749 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 31 12:19:50.804749 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 31 12:19:50.804749 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 31 12:19:50.804749 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:19:50.804749 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 31/61] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Oct 31 12:19:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:19:50.805887 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb9d096-f491-4159-bb5c-6ccc7cc5d615 None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:50.806111 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb9d096-f491-4159-bb5c-6ccc7cc5d615 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:50.806380 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb9d096-f491-4159-bb5c-6ccc7cc5d615 None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:50.808238 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 32/62] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:19:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:19:50.809327 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-5b2f9a56-b62c-48cb-822a-1224ad1cddd8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 31 12:19:50.809327 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:19:50.809327 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:19:50.809327 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:19:50.809327 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:19:50.809327 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:19:50.809327 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:19:50.809327 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:19:50.809327 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:19:50.809327 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:19:50.809327 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:19:50.809327 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:19:50.809327 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 31 12:19:50.809327 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 31 12:19:50.809327 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 31 12:19:50.809327 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 31 12:19:50.810155 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 31 12:19:50.810155 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 31 12:19:50.810155 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 31 12:19:50.810155 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 31 12:19:50.810155 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:19:50.810155 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:19:50.810155 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:19:50.810155 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:19:50.810155 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:19:50.810155 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:19:50.810155 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:19:50.810155 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 31 12:19:50.810155 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:19:50.810155 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:19:50.810155 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:19:50.810155 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:19:50.816031 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:19:50.816031 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:19:50.816031 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:19:50.816031 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:19:50.816031 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 31 12:19:50.816031 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 31 12:19:50.816031 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 31 12:19:50.816031 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 31 12:19:50.816031 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 31 12:19:50.816031 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 31 12:19:50.816031 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 31 12:19:50.816031 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:19:50.816031 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 31/63] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Oct 31 12:19:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:19:50.816031 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2f9a56-b62c-48cb-822a-1224ad1cddd8 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:19:50.816789 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95458f9b-65b1-4274-b782-244115840a36 None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:50.816789 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95458f9b-65b1-4274-b782-244115840a36 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:50.816789 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95458f9b-65b1-4274-b782-244115840a36 None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:50.816789 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 33/64] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:19:50 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:19:50.816789 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015076c3-b167-4ee7-ae4c-4085c71099f3 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:50.817062 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015076c3-b167-4ee7-ae4c-4085c71099f3 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:50.817442 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015076c3-b167-4ee7-ae4c-4085c71099f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:50.836830 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-015076c3-b167-4ee7-ae4c-4085c71099f3 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:19:50.838019 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-015076c3-b167-4ee7-ae4c-4085c71099f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:19:50.865804 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:19:50.907787 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 34/65] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:19:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 31 12:19:50.908553 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015076c3-b167-4ee7-ae4c-4085c71099f3 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:19:50.927200 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2f9a56-b62c-48cb-822a-1224ad1cddd8 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:19:50.968506 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef8f214-2f29-41bb-9000-80b7b29d981d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:19:50.969124 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef8f214-2f29-41bb-9000-80b7b29d981d None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:50.969404 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef8f214-2f29-41bb-9000-80b7b29d981d None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:50.969690 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef8f214-2f29-41bb-9000-80b7b29d981d None admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:50.970329 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef8f214-2f29-41bb-9000-80b7b29d981d None admin] RBAC: Adding query filter params (name=member) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:19:50.970634 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef8f214-2f29-41bb-9000-80b7b29d981d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:19:50.972034 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef8f214-2f29-41bb-9000-80b7b29d981d None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:19:50.984106 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 32/66] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Oct 31 12:19:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 31 12:19:50.985259 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4184dd-7e30-425f-9e26-c495240ba31b None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:50.985544 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4184dd-7e30-425f-9e26-c495240ba31b None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:50.985814 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4184dd-7e30-425f-9e26-c495240ba31b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:51.002533 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:19:51.003837 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-0b4184dd-7e30-425f-9e26-c495240ba31b None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:19:51.005163 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-0b4184dd-7e30-425f-9e26-c495240ba31b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:19:51.019461 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015076c3-b167-4ee7-ae4c-4085c71099f3 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:19:51.061526 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81abffda-6895-4e4c-b05d-c35dd2dae8a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:19:51.062136 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81abffda-6895-4e4c-b05d-c35dd2dae8a4 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:51.062415 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81abffda-6895-4e4c-b05d-c35dd2dae8a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:51.062687 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81abffda-6895-4e4c-b05d-c35dd2dae8a4 None admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:51.063386 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81abffda-6895-4e4c-b05d-c35dd2dae8a4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:19:51.063690 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81abffda-6895-4e4c-b05d-c35dd2dae8a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:19:51.065057 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81abffda-6895-4e4c-b05d-c35dd2dae8a4 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:19:51.078282 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 35/67] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Oct 31 12:19:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:19:51.078282 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81abffda-6895-4e4c-b05d-c35dd2dae8a4 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:19:51.091261 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 33/68] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:19:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 31 12:19:51.091261 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4184dd-7e30-425f-9e26-c495240ba31b None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:19:51.134839 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:19:51.206904 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4184dd-7e30-425f-9e26-c495240ba31b None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:19:51.214499 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81abffda-6895-4e4c-b05d-c35dd2dae8a4 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:19:51.245101 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9903f7-afa7-4a6c-a74b-b07ec2d77023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:19:51.245733 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9903f7-afa7-4a6c-a74b-b07ec2d77023 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:51.246013 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9903f7-afa7-4a6c-a74b-b07ec2d77023 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:51.246320 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9903f7-afa7-4a6c-a74b-b07ec2d77023 None admin] PATH_INFO: `/v3/users/d5915aed320e40a1ba7723089db18636` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:51.250111 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d4e3ed-a784-43e5-91cc-a4e2496ce12a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:19:51.250743 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d4e3ed-a784-43e5-91cc-a4e2496ce12a None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:51.251026 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d4e3ed-a784-43e5-91cc-a4e2496ce12a None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:51.251343 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d4e3ed-a784-43e5-91cc-a4e2496ce12a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:51.262643 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9903f7-afa7-4a6c-a74b-b07ec2d77023 None admin] RBAC: Authorizing `identity:get_user(user_id=d5915aed320e40a1ba7723089db18636)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:19:51.262783 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d4e3ed-a784-43e5-91cc-a4e2496ce12a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 31 12:19:51.264224 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9903f7-afa7-4a6c-a74b-b07ec2d77023 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:19:51.271205 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 34/69] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Oct 31 12:19:51 2022] GET /identity/v3/users/d5915aed320e40a1ba7723089db18636 => generated 275 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:19:51.271205 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9903f7-afa7-4a6c-a74b-b07ec2d77023 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:19:51.271418 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-c4d4e3ed-a784-43e5-91cc-a4e2496ce12a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 31 12:19:51.271418 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:19:51.271418 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:19:51.271418 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:19:51.271418 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:19:51.271418 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:19:51.271418 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:19:51.271418 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:19:51.271418 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:19:51.271418 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:19:51.271418 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:19:51.271418 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:19:51.271418 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 31 12:19:51.271418 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 31 12:19:51.271418 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 31 12:19:51.271418 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 31 12:19:51.272184 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 31 12:19:51.272184 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 31 12:19:51.272184 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 31 12:19:51.272184 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 31 12:19:51.272184 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:19:51.272184 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:19:51.272184 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:19:51.272184 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:19:51.272184 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:19:51.272184 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:19:51.272184 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:19:51.272184 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:19:51.272184 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:19:51.272184 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:19:51.272184 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:19:51.272184 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:19:51.272920 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:19:51.272920 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:19:51.272920 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:19:51.272920 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:19:51.272920 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 31 12:19:51.272920 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 31 12:19:51.272920 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 31 12:19:51.272920 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 31 12:19:51.272920 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 31 12:19:51.272920 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 31 12:19:51.272920 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 31 12:19:51.272920 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:19:51.272920 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 36/70] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Oct 31 12:19:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:19:51.272920 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d4e3ed-a784-43e5-91cc-a4e2496ce12a None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:19:51.286815 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:19:51.375535 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d4e3ed-a784-43e5-91cc-a4e2496ce12a None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:19:51.375916 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:19:51.385878 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9903f7-afa7-4a6c-a74b-b07ec2d77023 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:19:51.408855 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29dde4f-c421-4ef5-931d-4b53b6e22d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:19:51.409481 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29dde4f-c421-4ef5-931d-4b53b6e22d53 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:51.409758 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29dde4f-c421-4ef5-931d-4b53b6e22d53 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:51.410027 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29dde4f-c421-4ef5-931d-4b53b6e22d53 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:51.411123 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29dde4f-c421-4ef5-931d-4b53b6e22d53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:19:51.412579 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29dde4f-c421-4ef5-931d-4b53b6e22d53 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:19:51.414125 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 37/71] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Oct 31 12:19:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:19:51.416011 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29dde4f-c421-4ef5-931d-4b53b6e22d53 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:19:51.419953 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50e46ed-b8d9-41c9-b278-3df7245ec091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:19:51.420587 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50e46ed-b8d9-41c9-b278-3df7245ec091 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:51.420867 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50e46ed-b8d9-41c9-b278-3df7245ec091 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:51.421136 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50e46ed-b8d9-41c9-b278-3df7245ec091 None admin] PATH_INFO: `/v3/users/d2ddf15ba0c7488b8e49f974bba93de7` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:51.422260 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50e46ed-b8d9-41c9-b278-3df7245ec091 None admin] RBAC: Authorizing `identity:get_user(user_id=d2ddf15ba0c7488b8e49f974bba93de7)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:19:51.423717 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50e46ed-b8d9-41c9-b278-3df7245ec091 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:19:51.425303 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 35/72] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Oct 31 12:19:51 2022] GET /identity/v3/users/d2ddf15ba0c7488b8e49f974bba93de7 => generated 247 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:19:51.426043 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50e46ed-b8d9-41c9-b278-3df7245ec091 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:19:51.498718 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:19:51.521534 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29dde4f-c421-4ef5-931d-4b53b6e22d53 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:19:51.525969 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50e46ed-b8d9-41c9-b278-3df7245ec091 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:19:51.554660 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3335c96c-6a1e-4f9d-869e-5356c3b11a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:19:51.555316 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3335c96c-6a1e-4f9d-869e-5356c3b11a86 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:51.555595 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3335c96c-6a1e-4f9d-869e-5356c3b11a86 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:51.555875 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3335c96c-6a1e-4f9d-869e-5356c3b11a86 None admin] PATH_INFO: `/v3/projects/9bc6131ae0634994bd48138d77878576` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:51.557496 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3335c96c-6a1e-4f9d-869e-5356c3b11a86 None admin] RBAC: Authorizing `identity:get_project(project_id=9bc6131ae0634994bd48138d77878576)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:19:51.559094 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3335c96c-6a1e-4f9d-869e-5356c3b11a86 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:19:51.559848 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d941be5d-2a11-4e52-8c3d-2febe5bf9b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:19:51.560450 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d941be5d-2a11-4e52-8c3d-2febe5bf9b30 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:51.560950 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d941be5d-2a11-4e52-8c3d-2febe5bf9b30 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:51.561062 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 38/73] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Oct 31 12:19:51 2022] GET /identity/v3/projects/9bc6131ae0634994bd48138d77878576 => generated 298 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:19:51.561131 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d941be5d-2a11-4e52-8c3d-2febe5bf9b30 None admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:51.561489 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3335c96c-6a1e-4f9d-869e-5356c3b11a86 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:19:51.561644 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d941be5d-2a11-4e52-8c3d-2febe5bf9b30 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:19:51.561947 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d941be5d-2a11-4e52-8c3d-2febe5bf9b30 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:19:51.563371 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d941be5d-2a11-4e52-8c3d-2febe5bf9b30 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:19:51.572361 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 36/74] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Oct 31 12:19:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:19:51.573458 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d941be5d-2a11-4e52-8c3d-2febe5bf9b30 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:19:51.597783 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:19:51.664411 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3335c96c-6a1e-4f9d-869e-5356c3b11a86 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:19:51.672249 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d941be5d-2a11-4e52-8c3d-2febe5bf9b30 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:19:51.698261 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74469892-42b7-40d6-8808-54b51f212e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:19:51.698876 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74469892-42b7-40d6-8808-54b51f212e09 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:51.699154 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74469892-42b7-40d6-8808-54b51f212e09 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:51.699449 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74469892-42b7-40d6-8808-54b51f212e09 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:51.704318 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04068dd-f40f-436e-8f70-845a03c4300f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:19:51.704920 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04068dd-f40f-436e-8f70-845a03c4300f None admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:51.705202 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04068dd-f40f-436e-8f70-845a03c4300f None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:51.705476 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04068dd-f40f-436e-8f70-845a03c4300f None admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:51.706114 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04068dd-f40f-436e-8f70-845a03c4300f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:19:51.707561 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04068dd-f40f-436e-8f70-845a03c4300f None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:19:51.709141 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74469892-42b7-40d6-8808-54b51f212e09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 31 12:19:51.714899 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-74469892-42b7-40d6-8808-54b51f212e09 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 31 12:19:51.714899 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:19:51.714899 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:19:51.714899 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:19:51.714899 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:19:51.714899 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:19:51.714899 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:19:51.714899 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:19:51.714899 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:19:51.714899 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:19:51.714899 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:19:51.714899 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:19:51.714899 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 31 12:19:51.714899 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 31 12:19:51.714899 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 31 12:19:51.714899 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 31 12:19:51.715812 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 31 12:19:51.715812 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 31 12:19:51.715812 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 31 12:19:51.715812 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 31 12:19:51.715812 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:19:51.715812 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:19:51.715812 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:19:51.715812 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:19:51.715812 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:19:51.715812 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:19:51.715812 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:19:51.715812 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:19:51.715812 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:19:51.715812 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:19:51.715812 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:19:51.715812 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:19:51.716570 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:19:51.716570 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:19:51.716570 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:19:51.716570 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:19:51.716570 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 31 12:19:51.716570 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 31 12:19:51.716570 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 31 12:19:51.716570 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 31 12:19:51.716570 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 31 12:19:51.716570 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 31 12:19:51.716570 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 31 12:19:51.716570 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:19:51.716570 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 39/75] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Oct 31 12:19:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:19:51.717230 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74469892-42b7-40d6-8808-54b51f212e09 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:19:51.726949 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:19:51.743193 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 37/76] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Oct 31 12:19:51 2022] POST /identity/v3/users => generated 289 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:19:51.743193 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04068dd-f40f-436e-8f70-845a03c4300f None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:19:51.832291 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74469892-42b7-40d6-8808-54b51f212e09 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:19:51.835097 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:19:51.860235 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04068dd-f40f-436e-8f70-845a03c4300f None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:19:51.873192 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af447c15-3ed3-4240-aabd-ad10e5502cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:19:51.873844 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af447c15-3ed3-4240-aabd-ad10e5502cb0 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:51.874126 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af447c15-3ed3-4240-aabd-ad10e5502cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:51.874400 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af447c15-3ed3-4240-aabd-ad10e5502cb0 None admin] PATH_INFO: `/v3/projects/9bc6131ae0634994bd48138d77878576` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:51.875640 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af447c15-3ed3-4240-aabd-ad10e5502cb0 None admin] RBAC: Authorizing `identity:get_project(project_id=9bc6131ae0634994bd48138d77878576)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:19:51.877094 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af447c15-3ed3-4240-aabd-ad10e5502cb0 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:19:51.878779 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 40/77] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Oct 31 12:19:51 2022] GET /identity/v3/projects/9bc6131ae0634994bd48138d77878576 => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:19:51.884413 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af447c15-3ed3-4240-aabd-ad10e5502cb0 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:19:51.898578 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f69395d-f1ac-4982-8bf1-6870d3174b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:19:51.899283 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f69395d-f1ac-4982-8bf1-6870d3174b29 None admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:51.899582 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f69395d-f1ac-4982-8bf1-6870d3174b29 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:51.899844 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f69395d-f1ac-4982-8bf1-6870d3174b29 None admin] PATH_INFO: `/v3/projects/9bc6131ae0634994bd48138d77878576/users/d5915aed320e40a1ba7723089db18636/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:51.902446 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f69395d-f1ac-4982-8bf1-6870d3174b29 None admin] RBAC: Authorizing `identity:create_grant(project_id=9bc6131ae0634994bd48138d77878576, user_id=d5915aed320e40a1ba7723089db18636, role_id=ca1a5695f8bd499aa0b8cedaeecd2cc6)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:19:51.903917 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f69395d-f1ac-4982-8bf1-6870d3174b29 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:19:51.920896 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 38/78] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Oct 31 12:19:51 2022] PUT /identity/v3/projects/9bc6131ae0634994bd48138d77878576/users/d5915aed320e40a1ba7723089db18636/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:19:51.921264 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f69395d-f1ac-4982-8bf1-6870d3174b29 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:19:51.992124 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:19:52.033302 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af447c15-3ed3-4240-aabd-ad10e5502cb0 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:19:52.071893 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f69395d-f1ac-4982-8bf1-6870d3174b29 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:19:52.077604 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7cb684-4397-41ff-a9c3-3241b086faa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:19:52.078258 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7cb684-4397-41ff-a9c3-3241b086faa8 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:52.078542 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7cb684-4397-41ff-a9c3-3241b086faa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:52.078820 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7cb684-4397-41ff-a9c3-3241b086faa8 None admin] PATH_INFO: `/v3/users/d5915aed320e40a1ba7723089db18636` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:52.080551 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7cb684-4397-41ff-a9c3-3241b086faa8 None admin] RBAC: Authorizing `identity:get_user(user_id=d5915aed320e40a1ba7723089db18636)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:19:52.081495 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7cb684-4397-41ff-a9c3-3241b086faa8 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:19:52.083422 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 41/79] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Oct 31 12:19:51 2022] GET /identity/v3/users/d5915aed320e40a1ba7723089db18636 => generated 275 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:19:52.083912 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7cb684-4397-41ff-a9c3-3241b086faa8 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:19:52.117296 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f42d401-7075-4d83-a838-aa3aef6ad4e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:19:52.117945 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f42d401-7075-4d83-a838-aa3aef6ad4e4 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:52.118188 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f42d401-7075-4d83-a838-aa3aef6ad4e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:52.118461 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f42d401-7075-4d83-a838-aa3aef6ad4e4 None admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:52.119121 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f42d401-7075-4d83-a838-aa3aef6ad4e4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:19:52.120063 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f42d401-7075-4d83-a838-aa3aef6ad4e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:19:52.121625 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f42d401-7075-4d83-a838-aa3aef6ad4e4 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:19:52.129345 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:19:52.134277 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 39/80] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Oct 31 12:19:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:19:52.135520 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f42d401-7075-4d83-a838-aa3aef6ad4e4 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:19:52.214971 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7cb684-4397-41ff-a9c3-3241b086faa8 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:19:52.257622 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c21611-431d-42ba-a661-63587171d339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:19:52.258411 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c21611-431d-42ba-a661-63587171d339 None admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:52.258757 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c21611-431d-42ba-a661-63587171d339 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:52.259097 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c21611-431d-42ba-a661-63587171d339 None admin] PATH_INFO: `/v3/projects/9bc6131ae0634994bd48138d77878576/users/d2ddf15ba0c7488b8e49f974bba93de7/roles/176c07b4fe3b4537bdfeb4be21b379c5` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:52.262192 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c21611-431d-42ba-a661-63587171d339 None admin] RBAC: Authorizing `identity:create_grant(project_id=9bc6131ae0634994bd48138d77878576, user_id=d2ddf15ba0c7488b8e49f974bba93de7, role_id=176c07b4fe3b4537bdfeb4be21b379c5)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:19:52.263730 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c21611-431d-42ba-a661-63587171d339 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:19:52.265248 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f42d401-7075-4d83-a838-aa3aef6ad4e4 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:19:52.283019 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 42/81] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Oct 31 12:19:52 2022] PUT /identity/v3/projects/9bc6131ae0634994bd48138d77878576/users/d2ddf15ba0c7488b8e49f974bba93de7/roles/176c07b4fe3b4537bdfeb4be21b379c5 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:19:52.284156 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c21611-431d-42ba-a661-63587171d339 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:19:52.287286 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:19:52.330157 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae41f13-337e-4df5-a1b2-cd31a953d8cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:19:52.330829 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae41f13-337e-4df5-a1b2-cd31a953d8cf None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:52.331106 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae41f13-337e-4df5-a1b2-cd31a953d8cf None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:52.331411 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae41f13-337e-4df5-a1b2-cd31a953d8cf None admin] PATH_INFO: `/v3/projects/9bc6131ae0634994bd48138d77878576` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:52.333179 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae41f13-337e-4df5-a1b2-cd31a953d8cf None admin] RBAC: Authorizing `identity:get_project(project_id=9bc6131ae0634994bd48138d77878576)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:19:52.340075 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae41f13-337e-4df5-a1b2-cd31a953d8cf None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:19:52.345593 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 40/82] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Oct 31 12:19:52 2022] GET /identity/v3/projects/9bc6131ae0634994bd48138d77878576 => generated 298 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:19:52.351342 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae41f13-337e-4df5-a1b2-cd31a953d8cf None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:19:52.459025 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:19:52.475124 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c21611-431d-42ba-a661-63587171d339 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:19:52.500480 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae41f13-337e-4df5-a1b2-cd31a953d8cf None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:19:52.542078 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cccdf1-838f-4de9-ac5c-3b9fa70106e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:19:52.542592 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b3caf6-07d5-4532-8c91-d7bb498ff98b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:19:52.542709 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cccdf1-838f-4de9-ac5c-3b9fa70106e0 None admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:52.543078 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cccdf1-838f-4de9-ac5c-3b9fa70106e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:52.543467 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cccdf1-838f-4de9-ac5c-3b9fa70106e0 None admin] PATH_INFO: `/v3/projects/9bc6131ae0634994bd48138d77878576/users/d5915aed320e40a1ba7723089db18636/roles/65728d8bb0e546788426da322c34baba` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:52.543866 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b3caf6-07d5-4532-8c91-d7bb498ff98b None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:52.544411 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b3caf6-07d5-4532-8c91-d7bb498ff98b None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:52.544944 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b3caf6-07d5-4532-8c91-d7bb498ff98b None admin] PATH_INFO: `/v3/users/d5915aed320e40a1ba7723089db18636` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:52.546045 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cccdf1-838f-4de9-ac5c-3b9fa70106e0 None admin] RBAC: Authorizing `identity:create_grant(project_id=9bc6131ae0634994bd48138d77878576, user_id=d5915aed320e40a1ba7723089db18636, role_id=65728d8bb0e546788426da322c34baba)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:19:52.546518 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b3caf6-07d5-4532-8c91-d7bb498ff98b None admin] RBAC: Authorizing `identity:get_user(user_id=d5915aed320e40a1ba7723089db18636)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:19:52.553941 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cccdf1-838f-4de9-ac5c-3b9fa70106e0 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:19:52.554125 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b3caf6-07d5-4532-8c91-d7bb498ff98b None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:19:52.554125 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 43/83] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Oct 31 12:19:52 2022] GET /identity/v3/users/d5915aed320e40a1ba7723089db18636 => generated 275 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:19:52.562450 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b3caf6-07d5-4532-8c91-d7bb498ff98b None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:19:52.572400 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 41/84] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Oct 31 12:19:52 2022] PUT /identity/v3/projects/9bc6131ae0634994bd48138d77878576/users/d5915aed320e40a1ba7723089db18636/roles/65728d8bb0e546788426da322c34baba => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:19:52.661944 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:19:52.700843 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b3caf6-07d5-4532-8c91-d7bb498ff98b None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:19:52.743961 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a64c08c-773a-4ca5-9ee0-f122f255df92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:19:52.744798 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a64c08c-773a-4ca5-9ee0-f122f255df92 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:52.745444 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a64c08c-773a-4ca5-9ee0-f122f255df92 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:52.745444 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a64c08c-773a-4ca5-9ee0-f122f255df92 None admin] PATH_INFO: `/v3/projects/5cb588f8521c4eed8837380aa4af4156` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:52.746631 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a64c08c-773a-4ca5-9ee0-f122f255df92 None admin] RBAC: Authorizing `identity:get_project(project_id=5cb588f8521c4eed8837380aa4af4156)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:19:52.748193 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a64c08c-773a-4ca5-9ee0-f122f255df92 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:19:52.750257 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 44/85] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Oct 31 12:19:52 2022] GET /identity/v3/projects/5cb588f8521c4eed8837380aa4af4156 => generated 312 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:19:52.756334 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cccdf1-838f-4de9-ac5c-3b9fa70106e0 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:19:52.880726 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cccdf1-838f-4de9-ac5c-3b9fa70106e0 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:19:52.913002 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:19:52.927877 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27b33c8-d80d-4454-a4e1-f91979ce7125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:19:52.928616 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27b33c8-d80d-4454-a4e1-f91979ce7125 None admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:52.928899 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27b33c8-d80d-4454-a4e1-f91979ce7125 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:52.929176 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27b33c8-d80d-4454-a4e1-f91979ce7125 None admin] PATH_INFO: `/v3/projects/5cb588f8521c4eed8837380aa4af4156/users/d5915aed320e40a1ba7723089db18636/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:52.931270 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27b33c8-d80d-4454-a4e1-f91979ce7125 None admin] RBAC: Authorizing `identity:create_grant(project_id=5cb588f8521c4eed8837380aa4af4156, user_id=d5915aed320e40a1ba7723089db18636, role_id=ca1a5695f8bd499aa0b8cedaeecd2cc6)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:19:52.933310 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27b33c8-d80d-4454-a4e1-f91979ce7125 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:19:52.961020 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 42/86] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Oct 31 12:19:52 2022] PUT /identity/v3/projects/5cb588f8521c4eed8837380aa4af4156/users/d5915aed320e40a1ba7723089db18636/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 31 12:19:54.776446 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e03693f-cd57-4dda-aa77-043ba1496ee9 None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:54.777009 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e03693f-cd57-4dda-aa77-043ba1496ee9 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:54.777009 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e03693f-cd57-4dda-aa77-043ba1496ee9 None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:54.778517 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 45/87] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:19:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:19:54.783572 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebdcf06-260d-4500-853d-60eabf2a36d9 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:54.783960 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebdcf06-260d-4500-853d-60eabf2a36d9 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:54.784136 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebdcf06-260d-4500-853d-60eabf2a36d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:54.806669 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-aebdcf06-260d-4500-853d-60eabf2a36d9 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:19:54.808553 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-aebdcf06-260d-4500-853d-60eabf2a36d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:19:54.889360 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 43/88] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:19:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 31 12:19:55.249068 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c337fc-be8c-4bee-9a85-0a9588eb77c2 None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:55.249250 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c337fc-be8c-4bee-9a85-0a9588eb77c2 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:55.249530 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c337fc-be8c-4bee-9a85-0a9588eb77c2 None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:55.251090 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 46/89] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:19:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:19:55.256647 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559aa2a6-a94a-40b1-a934-78c84cc5bf5d None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:55.256936 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559aa2a6-a94a-40b1-a934-78c84cc5bf5d None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:55.257209 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559aa2a6-a94a-40b1-a934-78c84cc5bf5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:55.283271 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-559aa2a6-a94a-40b1-a934-78c84cc5bf5d None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:19:55.284473 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-559aa2a6-a94a-40b1-a934-78c84cc5bf5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:19:55.376135 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 44/90] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:19:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 31 12:19:55.417710 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dd9d63-7fcd-4f04-946c-cb1355cf4b61 None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:55.418007 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dd9d63-7fcd-4f04-946c-cb1355cf4b61 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:55.418270 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dd9d63-7fcd-4f04-946c-cb1355cf4b61 None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:55.419667 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 47/91] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:19:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:19:55.425114 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142f1f3e-08f6-454f-8a4d-cf406b4be31c None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:55.425406 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142f1f3e-08f6-454f-8a4d-cf406b4be31c None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:55.425681 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142f1f3e-08f6-454f-8a4d-cf406b4be31c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:55.438407 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dd9d63-7fcd-4f04-946c-cb1355cf4b61 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:19:55.454404 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-142f1f3e-08f6-454f-8a4d-cf406b4be31c None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:19:55.455621 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-142f1f3e-08f6-454f-8a4d-cf406b4be31c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:19:55.532434 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:19:55.575179 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 45/92] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:19:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 31 12:19:55.576717 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5592a48d-0433-4acf-a04f-bd17ecb23876 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:55.577002 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5592a48d-0433-4acf-a04f-bd17ecb23876 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:55.577274 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5592a48d-0433-4acf-a04f-bd17ecb23876 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:55.578513 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 46/93] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:19:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:19:55.584394 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e050383d-d2db-4e1a-8cae-d07f43c3cc32 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:55.584675 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e050383d-d2db-4e1a-8cae-d07f43c3cc32 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:55.584952 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e050383d-d2db-4e1a-8cae-d07f43c3cc32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:55.621934 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-e050383d-d2db-4e1a-8cae-d07f43c3cc32 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:19:55.623120 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-e050383d-d2db-4e1a-8cae-d07f43c3cc32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:19:55.625984 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dd9d63-7fcd-4f04-946c-cb1355cf4b61 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:19:55.684192 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9319cd6f-0c77-43f1-8ddd-470928bc17f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:19:55.684840 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9319cd6f-0c77-43f1-8ddd-470928bc17f5 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:55.685110 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9319cd6f-0c77-43f1-8ddd-470928bc17f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:55.685660 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9319cd6f-0c77-43f1-8ddd-470928bc17f5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:55.698181 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:19:55.703172 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9319cd6f-0c77-43f1-8ddd-470928bc17f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 31 12:19:55.712337 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-9319cd6f-0c77-43f1-8ddd-470928bc17f5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 31 12:19:55.712337 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:19:55.712337 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:19:55.712337 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:19:55.712337 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:19:55.712337 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:19:55.712337 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:19:55.712337 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:19:55.712337 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:19:55.712337 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:19:55.712337 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:19:55.712337 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:19:55.712337 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 31 12:19:55.712337 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 31 12:19:55.712337 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 31 12:19:55.712337 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 31 12:19:55.713457 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 31 12:19:55.713457 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 31 12:19:55.713457 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 31 12:19:55.713457 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 31 12:19:55.713457 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:19:55.713457 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:19:55.713457 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:19:55.713457 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:19:55.713457 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:19:55.713457 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:19:55.713457 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:19:55.713457 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:19:55.713457 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:19:55.713457 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:19:55.713457 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:19:55.713457 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:19:55.714226 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:19:55.714226 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:19:55.714226 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:19:55.714226 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:19:55.714226 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 31 12:19:55.714226 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 31 12:19:55.714226 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 31 12:19:55.714226 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 31 12:19:55.714226 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 31 12:19:55.714226 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 31 12:19:55.714226 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 31 12:19:55.714226 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:19:55.718192 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 48/94] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Oct 31 12:19:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:19:55.719022 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9319cd6f-0c77-43f1-8ddd-470928bc17f5 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:19:55.749504 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 47/95] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:19:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 31 12:19:55.750925 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e050383d-d2db-4e1a-8cae-d07f43c3cc32 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:19:55.844060 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9319cd6f-0c77-43f1-8ddd-470928bc17f5 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:19:55.879890 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68af1bdf-1ae2-44ab-86ed-0100182b4efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:19:55.880517 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af1bdf-1ae2-44ab-86ed-0100182b4efc None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:55.880796 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af1bdf-1ae2-44ab-86ed-0100182b4efc None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:55.881069 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af1bdf-1ae2-44ab-86ed-0100182b4efc None admin] PATH_INFO: `/v3/roles/member` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:55.892022 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68af1bdf-1ae2-44ab-86ed-0100182b4efc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 31 12:19:55.896216 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e050383d-d2db-4e1a-8cae-d07f43c3cc32 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:19:55.897754 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-68af1bdf-1ae2-44ab-86ed-0100182b4efc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 31 12:19:55.897754 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:19:55.897754 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:19:55.897754 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:19:55.897754 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:19:55.897754 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:19:55.897754 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:19:55.897754 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:19:55.897754 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:19:55.897754 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:19:55.897754 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:19:55.897754 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:19:55.897754 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 31 12:19:55.897754 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 31 12:19:55.897754 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 31 12:19:55.897754 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 31 12:19:55.899789 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 31 12:19:55.899789 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 31 12:19:55.899789 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 31 12:19:55.899789 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 31 12:19:55.899789 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:19:55.899789 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:19:55.899789 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:19:55.899789 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:19:55.899789 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:19:55.899789 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:19:55.899789 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:19:55.899789 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:19:55.899789 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:19:55.899789 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:19:55.899789 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:19:55.899789 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:19:55.901793 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:19:55.901793 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:19:55.901793 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:19:55.901793 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:19:55.901793 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 31 12:19:55.901793 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 31 12:19:55.901793 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 31 12:19:55.901793 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 31 12:19:55.901793 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 31 12:19:55.901793 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 31 12:19:55.901793 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 31 12:19:55.901793 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:19:55.901793 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 49/96] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Oct 31 12:19:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:19:55.910027 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68af1bdf-1ae2-44ab-86ed-0100182b4efc None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:19:55.931529 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:19:55.986228 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb727aa-8ad0-4f42-8216-30d0269b7343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:19:55.986851 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb727aa-8ad0-4f42-8216-30d0269b7343 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:55.987126 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb727aa-8ad0-4f42-8216-30d0269b7343 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:55.988739 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb727aa-8ad0-4f42-8216-30d0269b7343 None admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:55.989420 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb727aa-8ad0-4f42-8216-30d0269b7343 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:19:55.989727 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb727aa-8ad0-4f42-8216-30d0269b7343 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:19:55.991114 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb727aa-8ad0-4f42-8216-30d0269b7343 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:19:56.009870 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 48/97] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Oct 31 12:19:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:19:56.031054 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb727aa-8ad0-4f42-8216-30d0269b7343 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:19:56.031708 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:19:56.076883 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68af1bdf-1ae2-44ab-86ed-0100182b4efc None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:19:56.111594 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94dbc22-0cd2-4d05-b8f4-2aca54424c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:19:56.112127 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94dbc22-0cd2-4d05-b8f4-2aca54424c7a None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:56.112406 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94dbc22-0cd2-4d05-b8f4-2aca54424c7a None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:56.112677 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94dbc22-0cd2-4d05-b8f4-2aca54424c7a None admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:56.113338 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94dbc22-0cd2-4d05-b8f4-2aca54424c7a None admin] RBAC: Adding query filter params (name=member) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:19:56.113638 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94dbc22-0cd2-4d05-b8f4-2aca54424c7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:19:56.115010 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94dbc22-0cd2-4d05-b8f4-2aca54424c7a None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:19:56.124835 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 50/98] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Oct 31 12:19:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:19:56.125837 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94dbc22-0cd2-4d05-b8f4-2aca54424c7a None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:19:56.189673 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb727aa-8ad0-4f42-8216-30d0269b7343 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:19:56.234178 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40580334-f17f-4756-8ca5-0dc8a5990a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:19:56.234990 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40580334-f17f-4756-8ca5-0dc8a5990a9b None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:56.235627 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40580334-f17f-4756-8ca5-0dc8a5990a9b None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:56.236037 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40580334-f17f-4756-8ca5-0dc8a5990a9b None admin] PATH_INFO: `/v3/users/f1215134839e4b60a3fb995403b82a9a` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:56.256088 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40580334-f17f-4756-8ca5-0dc8a5990a9b None admin] RBAC: Authorizing `identity:get_user(user_id=f1215134839e4b60a3fb995403b82a9a)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:19:56.257551 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40580334-f17f-4756-8ca5-0dc8a5990a9b None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:19:56.259440 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 49/99] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Oct 31 12:19:56 2022] GET /identity/v3/users/f1215134839e4b60a3fb995403b82a9a => generated 289 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:19:56.260988 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40580334-f17f-4756-8ca5-0dc8a5990a9b None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:19:56.271022 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94dbc22-0cd2-4d05-b8f4-2aca54424c7a None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:19:56.289721 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:19:56.294407 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:19:56.309546 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771f0eda-d072-431a-93df-637b30abb892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:19:56.310184 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771f0eda-d072-431a-93df-637b30abb892 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:56.310461 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771f0eda-d072-431a-93df-637b30abb892 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:56.310901 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771f0eda-d072-431a-93df-637b30abb892 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:56.321521 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771f0eda-d072-431a-93df-637b30abb892 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 31 12:19:56.327533 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-771f0eda-d072-431a-93df-637b30abb892 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 31 12:19:56.327533 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:19:56.327533 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:19:56.327533 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:19:56.327533 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:19:56.327533 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:19:56.327533 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:19:56.327533 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:19:56.327533 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:19:56.327533 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:19:56.327533 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:19:56.327533 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:19:56.327533 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 31 12:19:56.327533 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 31 12:19:56.327533 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 31 12:19:56.327533 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 31 12:19:56.328374 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 31 12:19:56.328374 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 31 12:19:56.328374 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 31 12:19:56.328374 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 31 12:19:56.328374 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:19:56.328374 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:19:56.328374 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:19:56.328374 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:19:56.328374 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:19:56.328374 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:19:56.328374 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:19:56.328374 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:19:56.328374 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:19:56.328374 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:19:56.328374 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:19:56.328374 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:19:56.329127 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:19:56.329127 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:19:56.329127 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:19:56.329127 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:19:56.329127 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 31 12:19:56.329127 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 31 12:19:56.329127 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 31 12:19:56.329127 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 31 12:19:56.329127 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 31 12:19:56.329127 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 31 12:19:56.329127 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 31 12:19:56.329127 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:19:56.329127 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 51/100] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Oct 31 12:19:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:19:56.329748 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771f0eda-d072-431a-93df-637b30abb892 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:19:56.372551 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40580334-f17f-4756-8ca5-0dc8a5990a9b None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:19:56.405483 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91f8258-cd45-46f3-987a-9e38dc297391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:19:56.406113 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91f8258-cd45-46f3-987a-9e38dc297391 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:56.406393 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91f8258-cd45-46f3-987a-9e38dc297391 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:56.406665 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91f8258-cd45-46f3-987a-9e38dc297391 None admin] PATH_INFO: `/v3/users/d5915aed320e40a1ba7723089db18636` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:56.407811 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91f8258-cd45-46f3-987a-9e38dc297391 None admin] RBAC: Authorizing `identity:get_user(user_id=d5915aed320e40a1ba7723089db18636)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:19:56.409236 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91f8258-cd45-46f3-987a-9e38dc297391 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:19:56.410805 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 50/101] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Oct 31 12:19:56 2022] GET /identity/v3/users/d5915aed320e40a1ba7723089db18636 => generated 275 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:19:56.417050 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1dc2ff-3008-42a2-8c02-bf7574802438 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:56.417345 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1dc2ff-3008-42a2-8c02-bf7574802438 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:56.417620 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1dc2ff-3008-42a2-8c02-bf7574802438 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:56.418876 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 51/102] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:19:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:19:56.419606 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1dc2ff-3008-42a2-8c02-bf7574802438 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:19:56.435450 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771f0eda-d072-431a-93df-637b30abb892 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:19:56.467961 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13bb30e-6824-41e5-8f5e-d1dfa410edbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:19:56.468582 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13bb30e-6824-41e5-8f5e-d1dfa410edbb None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:56.468874 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13bb30e-6824-41e5-8f5e-d1dfa410edbb None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:56.469164 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13bb30e-6824-41e5-8f5e-d1dfa410edbb None admin] PATH_INFO: `/v3/domains/default` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:56.470243 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13bb30e-6824-41e5-8f5e-d1dfa410edbb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:19:56.471698 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13bb30e-6824-41e5-8f5e-d1dfa410edbb None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:19:56.473254 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 52/103] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Oct 31 12:19:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:19:56.474005 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13bb30e-6824-41e5-8f5e-d1dfa410edbb None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:19:56.514284 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:19:56.521282 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1dc2ff-3008-42a2-8c02-bf7574802438 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:19:56.526381 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:19:56.554772 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08883026-0f70-498a-ba27-3aa746c7d481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:19:56.555433 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08883026-0f70-498a-ba27-3aa746c7d481 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:56.555710 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08883026-0f70-498a-ba27-3aa746c7d481 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:56.555987 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08883026-0f70-498a-ba27-3aa746c7d481 None admin] PATH_INFO: `/v3/projects/9bc6131ae0634994bd48138d77878576` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:56.557686 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08883026-0f70-498a-ba27-3aa746c7d481 None admin] RBAC: Authorizing `identity:get_project(project_id=9bc6131ae0634994bd48138d77878576)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:19:56.559112 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08883026-0f70-498a-ba27-3aa746c7d481 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:19:56.560946 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 52/104] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Oct 31 12:19:56 2022] GET /identity/v3/projects/9bc6131ae0634994bd48138d77878576 => generated 298 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:19:56.561876 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be8a3cc-ed82-471a-b673-92f00b1ab113 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:56.562159 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be8a3cc-ed82-471a-b673-92f00b1ab113 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:56.562431 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be8a3cc-ed82-471a-b673-92f00b1ab113 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:56.563654 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 53/105] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:19:56 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:19:56.564344 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be8a3cc-ed82-471a-b673-92f00b1ab113 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:19:56.586031 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13bb30e-6824-41e5-8f5e-d1dfa410edbb None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:19:56.620259 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33026ea-d850-4dba-9a8a-a8e27fbf3c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:19:56.620867 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33026ea-d850-4dba-9a8a-a8e27fbf3c9e None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:56.621152 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33026ea-d850-4dba-9a8a-a8e27fbf3c9e None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:56.621427 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33026ea-d850-4dba-9a8a-a8e27fbf3c9e None admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:56.622090 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33026ea-d850-4dba-9a8a-a8e27fbf3c9e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:19:56.622390 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33026ea-d850-4dba-9a8a-a8e27fbf3c9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:19:56.623867 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33026ea-d850-4dba-9a8a-a8e27fbf3c9e None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:19:56.631807 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 53/106] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Oct 31 12:19:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:19:56.632946 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7573d72-956d-4c3e-895d-fab8317f7683 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:56.633231 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7573d72-956d-4c3e-895d-fab8317f7683 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:56.633505 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7573d72-956d-4c3e-895d-fab8317f7683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:56.653251 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-b7573d72-956d-4c3e-895d-fab8317f7683 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:19:56.654463 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-b7573d72-956d-4c3e-895d-fab8317f7683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:19:56.677615 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be8a3cc-ed82-471a-b673-92f00b1ab113 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:19:56.721987 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdd4900-3659-4a94-987a-11ca816773ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:19:56.722763 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdd4900-3659-4a94-987a-11ca816773ad None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:56.723108 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdd4900-3659-4a94-987a-11ca816773ad None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:56.723560 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdd4900-3659-4a94-987a-11ca816773ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:56.732082 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 54/107] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:19:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 31 12:19:56.732719 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7573d72-956d-4c3e-895d-fab8317f7683 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:19:56.759052 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:19:56.784862 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:19:56.804496 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdd4900-3659-4a94-987a-11ca816773ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:19:56.806146 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdd4900-3659-4a94-987a-11ca816773ad None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:19:56.849253 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7573d72-956d-4c3e-895d-fab8317f7683 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:19:56.891645 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f1b980-ef01-4d54-a402-57a8d51ebdf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:19:56.893096 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f1b980-ef01-4d54-a402-57a8d51ebdf6 None admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:56.893518 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f1b980-ef01-4d54-a402-57a8d51ebdf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:56.893928 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f1b980-ef01-4d54-a402-57a8d51ebdf6 None admin] PATH_INFO: `/v3/projects` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:56.895154 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f1b980-ef01-4d54-a402-57a8d51ebdf6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:19:56.897018 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f1b980-ef01-4d54-a402-57a8d51ebdf6 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:19:56.925842 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 54/108] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Oct 31 12:19:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 31 12:19:56.926662 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdd4900-3659-4a94-987a-11ca816773ad None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:19:56.930469 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 55/109] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Oct 31 12:19:56 2022] POST /identity/v3/projects => generated 302 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:19:56.933731 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347e8ee7-39e6-4e7f-a085-8857d05bbc0b None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:56.934250 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347e8ee7-39e6-4e7f-a085-8857d05bbc0b None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:56.934745 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347e8ee7-39e6-4e7f-a085-8857d05bbc0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:56.971673 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-347e8ee7-39e6-4e7f-a085-8857d05bbc0b None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:19:56.974522 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-347e8ee7-39e6-4e7f-a085-8857d05bbc0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:19:57.038631 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:19:57.094982 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdd4900-3659-4a94-987a-11ca816773ad None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:19:57.103430 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 56/110] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:19:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 31 12:19:57.103430 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347e8ee7-39e6-4e7f-a085-8857d05bbc0b None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:19:57.123459 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:19:57.149675 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73744a4-bf49-4662-9efb-1a77221e076c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:19:57.150259 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73744a4-bf49-4662-9efb-1a77221e076c None admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:57.150536 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73744a4-bf49-4662-9efb-1a77221e076c None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:57.150990 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73744a4-bf49-4662-9efb-1a77221e076c None admin] PATH_INFO: `/v3/projects/9bc6131ae0634994bd48138d77878576/users/f1215134839e4b60a3fb995403b82a9a/roles/056e0848b1da43f1a1eaae44d5ecddc8` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:57.153734 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73744a4-bf49-4662-9efb-1a77221e076c None admin] RBAC: Authorizing `identity:create_grant(project_id=9bc6131ae0634994bd48138d77878576, user_id=f1215134839e4b60a3fb995403b82a9a, role_id=056e0848b1da43f1a1eaae44d5ecddc8)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:19:57.155974 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73744a4-bf49-4662-9efb-1a77221e076c None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:19:57.174223 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 55/111] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Oct 31 12:19:56 2022] PUT /identity/v3/projects/9bc6131ae0634994bd48138d77878576/users/f1215134839e4b60a3fb995403b82a9a/roles/056e0848b1da43f1a1eaae44d5ecddc8 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:19:57.175529 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73744a4-bf49-4662-9efb-1a77221e076c None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:19:57.228748 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347e8ee7-39e6-4e7f-a085-8857d05bbc0b None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:19:57.271504 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a159de-2765-41fc-91db-4013b7f367fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:19:57.272165 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a159de-2765-41fc-91db-4013b7f367fc None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:57.272496 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a159de-2765-41fc-91db-4013b7f367fc None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:57.272780 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a159de-2765-41fc-91db-4013b7f367fc None admin] PATH_INFO: `/v3/users/d2ddf15ba0c7488b8e49f974bba93de7` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:57.274097 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a159de-2765-41fc-91db-4013b7f367fc None admin] RBAC: Authorizing `identity:get_user(user_id=d2ddf15ba0c7488b8e49f974bba93de7)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:19:57.275668 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a159de-2765-41fc-91db-4013b7f367fc None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:19:57.277486 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 57/112] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Oct 31 12:19:57 2022] GET /identity/v3/users/d2ddf15ba0c7488b8e49f974bba93de7 => generated 247 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:19:57.278697 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a159de-2765-41fc-91db-4013b7f367fc None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:19:57.301729 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73744a4-bf49-4662-9efb-1a77221e076c None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:19:57.343637 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:19:57.351818 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db312a67-b734-46ad-bde4-919da9225279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:19:57.352829 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db312a67-b734-46ad-bde4-919da9225279 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:57.353335 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db312a67-b734-46ad-bde4-919da9225279 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:57.354855 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db312a67-b734-46ad-bde4-919da9225279 None admin] PATH_INFO: `/v3/projects/9bc6131ae0634994bd48138d77878576` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:57.358395 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db312a67-b734-46ad-bde4-919da9225279 None admin] RBAC: Authorizing `identity:get_project(project_id=9bc6131ae0634994bd48138d77878576)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:19:57.360178 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db312a67-b734-46ad-bde4-919da9225279 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:19:57.364553 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 56/113] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Oct 31 12:19:57 2022] GET /identity/v3/projects/9bc6131ae0634994bd48138d77878576 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:19:57.373983 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db312a67-b734-46ad-bde4-919da9225279 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:19:57.423642 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:19:57.433095 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a159de-2765-41fc-91db-4013b7f367fc None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:19:57.474509 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1b4c90-3709-4e90-b146-abedeac5d8fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:19:57.475064 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1b4c90-3709-4e90-b146-abedeac5d8fa None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:57.475392 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1b4c90-3709-4e90-b146-abedeac5d8fa None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:57.475673 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1b4c90-3709-4e90-b146-abedeac5d8fa None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:57.487287 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1b4c90-3709-4e90-b146-abedeac5d8fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 31 12:19:57.492317 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-0a1b4c90-3709-4e90-b146-abedeac5d8fa None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 31 12:19:57.492317 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:19:57.492317 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:19:57.492317 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:19:57.492317 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:19:57.492317 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:19:57.492317 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:19:57.492317 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:19:57.492317 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:19:57.492317 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:19:57.492317 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:19:57.492317 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:19:57.492317 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 31 12:19:57.492317 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 31 12:19:57.492317 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 31 12:19:57.492317 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 31 12:19:57.493176 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 31 12:19:57.493176 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 31 12:19:57.493176 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 31 12:19:57.493176 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 31 12:19:57.493176 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:19:57.493176 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:19:57.493176 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:19:57.493176 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:19:57.493176 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:19:57.493176 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:19:57.493176 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:19:57.493176 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:19:57.493176 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:19:57.493176 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:19:57.493176 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:19:57.493176 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:19:57.493921 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:19:57.493921 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:19:57.493921 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:19:57.493921 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:19:57.493921 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 31 12:19:57.493921 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 31 12:19:57.493921 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 31 12:19:57.493921 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 31 12:19:57.493921 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 31 12:19:57.493921 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 31 12:19:57.493921 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 31 12:19:57.493921 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:19:57.493921 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 58/114] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Oct 31 12:19:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:19:57.499331 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1b4c90-3709-4e90-b146-abedeac5d8fa None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:19:57.529962 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db312a67-b734-46ad-bde4-919da9225279 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:19:57.569833 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c0e496-b96a-44e5-9185-d7970f79b3d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:19:57.570747 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c0e496-b96a-44e5-9185-d7970f79b3d2 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:57.571222 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c0e496-b96a-44e5-9185-d7970f79b3d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:57.571676 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c0e496-b96a-44e5-9185-d7970f79b3d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:57.624044 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1b4c90-3709-4e90-b146-abedeac5d8fa None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:19:57.653394 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:19:57.664887 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c0e496-b96a-44e5-9185-d7970f79b3d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:19:57.667983 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c0e496-b96a-44e5-9185-d7970f79b3d2 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:19:57.670976 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76de8f4e-8c86-4e34-8d67-239130238b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:19:57.672654 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76de8f4e-8c86-4e34-8d67-239130238b60 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:57.673008 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76de8f4e-8c86-4e34-8d67-239130238b60 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:57.673356 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76de8f4e-8c86-4e34-8d67-239130238b60 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:57.674361 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76de8f4e-8c86-4e34-8d67-239130238b60 None admin] RBAC: Adding query filter params (role.id=ca1a5695f8bd499aa0b8cedaeecd2cc6, scope.project.id=9bc6131ae0634994bd48138d77878576, user.id=d5915aed320e40a1ba7723089db18636) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:19:57.674684 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76de8f4e-8c86-4e34-8d67-239130238b60 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:19:57.676258 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76de8f4e-8c86-4e34-8d67-239130238b60 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:19:57.695197 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 59/115] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Oct 31 12:19:57 2022] GET /identity/v3/role_assignments?user.id=d5915aed320e40a1ba7723089db18636&scope.project.id=9bc6131ae0634994bd48138d77878576&role.id=ca1a5695f8bd499aa0b8cedaeecd2cc6 => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:19:57.695197 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76de8f4e-8c86-4e34-8d67-239130238b60 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:19:57.707895 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:19:57.811254 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76de8f4e-8c86-4e34-8d67-239130238b60 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:19:57.849551 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b167480-7762-4152-b9a1-70cd08e73c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:19:57.850175 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b167480-7762-4152-b9a1-70cd08e73c8f None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:57.850460 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b167480-7762-4152-b9a1-70cd08e73c8f None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:57.850734 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b167480-7762-4152-b9a1-70cd08e73c8f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:57.863708 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b167480-7762-4152-b9a1-70cd08e73c8f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 31 12:19:57.870197 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-6b167480-7762-4152-b9a1-70cd08e73c8f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 31 12:19:57.870197 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:19:57.870197 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:19:57.870197 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:19:57.870197 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:19:57.870197 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:19:57.870197 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:19:57.870197 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:19:57.870197 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:19:57.870197 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:19:57.870197 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:19:57.870197 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:19:57.870197 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 31 12:19:57.870197 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 31 12:19:57.870197 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 31 12:19:57.870197 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 31 12:19:57.871339 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 31 12:19:57.871339 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 31 12:19:57.871339 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 31 12:19:57.871339 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 31 12:19:57.871339 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:19:57.871339 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:19:57.871339 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:19:57.871339 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:19:57.871339 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:19:57.871339 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:19:57.871339 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:19:57.871339 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:19:57.871339 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:19:57.871339 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:19:57.871339 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:19:57.871339 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:19:57.872138 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:19:57.872138 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:19:57.872138 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:19:57.872138 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:19:57.872138 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 31 12:19:57.872138 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 31 12:19:57.872138 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 31 12:19:57.872138 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 31 12:19:57.872138 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 31 12:19:57.872138 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 31 12:19:57.872138 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 31 12:19:57.872138 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:19:57.872138 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 60/116] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Oct 31 12:19:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:19:57.872762 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b167480-7762-4152-b9a1-70cd08e73c8f None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:19:57.897870 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 57/117] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Oct 31 12:19:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 531 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 31 12:19:57.899120 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c0e496-b96a-44e5-9185-d7970f79b3d2 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:19:58.017835 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:19:58.061862 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b167480-7762-4152-b9a1-70cd08e73c8f None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:19:58.079157 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:19:58.090087 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c0e496-b96a-44e5-9185-d7970f79b3d2 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:19:58.108892 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b90d3e4-f561-49af-a645-16879ca3dbf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:19:58.109524 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b90d3e4-f561-49af-a645-16879ca3dbf3 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:58.110134 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b90d3e4-f561-49af-a645-16879ca3dbf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:58.110444 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b90d3e4-f561-49af-a645-16879ca3dbf3 None admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:58.111402 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b90d3e4-f561-49af-a645-16879ca3dbf3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:19:58.111712 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b90d3e4-f561-49af-a645-16879ca3dbf3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:19:58.113088 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b90d3e4-f561-49af-a645-16879ca3dbf3 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:19:58.128187 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 61/118] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Oct 31 12:19:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:19:58.129304 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b90d3e4-f561-49af-a645-16879ca3dbf3 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:19:58.145171 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22aa9236-c4d9-412a-a9c7-ac5e2c842789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:19:58.145799 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aa9236-c4d9-412a-a9c7-ac5e2c842789 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:58.146083 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aa9236-c4d9-412a-a9c7-ac5e2c842789 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:58.146356 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aa9236-c4d9-412a-a9c7-ac5e2c842789 None admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:58.147025 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22aa9236-c4d9-412a-a9c7-ac5e2c842789 None admin] RBAC: Adding query filter params (name=member) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:19:58.147355 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22aa9236-c4d9-412a-a9c7-ac5e2c842789 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:19:58.148746 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22aa9236-c4d9-412a-a9c7-ac5e2c842789 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:19:58.160264 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 58/119] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Oct 31 12:19:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:19:58.161641 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22aa9236-c4d9-412a-a9c7-ac5e2c842789 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:19:58.261013 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b90d3e4-f561-49af-a645-16879ca3dbf3 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:19:58.278087 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22aa9236-c4d9-412a-a9c7-ac5e2c842789 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:19:58.300201 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de2912f-c12b-4976-b834-b96f4f975272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:19:58.300840 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de2912f-c12b-4976-b834-b96f4f975272 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:58.301134 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de2912f-c12b-4976-b834-b96f4f975272 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:58.301396 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de2912f-c12b-4976-b834-b96f4f975272 None admin] PATH_INFO: `/v3/projects/9bc6131ae0634994bd48138d77878576` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:58.302509 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de2912f-c12b-4976-b834-b96f4f975272 None admin] RBAC: Authorizing `identity:get_project(project_id=9bc6131ae0634994bd48138d77878576)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:19:58.303990 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de2912f-c12b-4976-b834-b96f4f975272 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:19:58.305873 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 62/120] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Oct 31 12:19:58 2022] GET /identity/v3/projects/9bc6131ae0634994bd48138d77878576 => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:19:58.310989 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de2912f-c12b-4976-b834-b96f4f975272 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:19:58.316023 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:19:58.326860 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fd883b-bbcc-48a8-9f21-4b4fe39f7dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:19:58.327810 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fd883b-bbcc-48a8-9f21-4b4fe39f7dae None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:58.328304 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fd883b-bbcc-48a8-9f21-4b4fe39f7dae None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:58.328767 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fd883b-bbcc-48a8-9f21-4b4fe39f7dae None admin] PATH_INFO: `/v3/users/d5915aed320e40a1ba7723089db18636` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:58.330166 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fd883b-bbcc-48a8-9f21-4b4fe39f7dae None admin] RBAC: Authorizing `identity:get_user(user_id=d5915aed320e40a1ba7723089db18636)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:19:58.331828 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fd883b-bbcc-48a8-9f21-4b4fe39f7dae None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:19:58.338883 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 59/121] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Oct 31 12:19:58 2022] GET /identity/v3/users/d5915aed320e40a1ba7723089db18636 => generated 275 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:19:58.340088 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fd883b-bbcc-48a8-9f21-4b4fe39f7dae None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:19:58.366822 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:19:58.421920 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de2912f-c12b-4976-b834-b96f4f975272 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:19:58.448695 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fd883b-bbcc-48a8-9f21-4b4fe39f7dae None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:19:58.458363 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1c01fd-fe7c-4246-8d45-acb4171d23c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:19:58.459006 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1c01fd-fe7c-4246-8d45-acb4171d23c8 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:58.459349 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1c01fd-fe7c-4246-8d45-acb4171d23c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:58.459624 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1c01fd-fe7c-4246-8d45-acb4171d23c8 None admin] PATH_INFO: `/v3/users/d5915aed320e40a1ba7723089db18636` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:58.460783 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1c01fd-fe7c-4246-8d45-acb4171d23c8 None admin] RBAC: Authorizing `identity:get_user(user_id=d5915aed320e40a1ba7723089db18636)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:19:58.462225 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1c01fd-fe7c-4246-8d45-acb4171d23c8 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:19:58.469025 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 63/122] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Oct 31 12:19:58 2022] GET /identity/v3/users/d5915aed320e40a1ba7723089db18636 => generated 275 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:19:58.469025 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1c01fd-fe7c-4246-8d45-acb4171d23c8 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:19:58.492706 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e92c1a5-cff6-4bb0-9e32-4439a0b13665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:19:58.493694 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e92c1a5-cff6-4bb0-9e32-4439a0b13665 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:58.494246 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e92c1a5-cff6-4bb0-9e32-4439a0b13665 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:58.495044 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e92c1a5-cff6-4bb0-9e32-4439a0b13665 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:58.496028 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e92c1a5-cff6-4bb0-9e32-4439a0b13665 None admin] RBAC: Adding query filter params (role.id=176c07b4fe3b4537bdfeb4be21b379c5, scope.project.id=9bc6131ae0634994bd48138d77878576, user.id=d2ddf15ba0c7488b8e49f974bba93de7) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:19:58.496479 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e92c1a5-cff6-4bb0-9e32-4439a0b13665 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:19:58.498041 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e92c1a5-cff6-4bb0-9e32-4439a0b13665 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:19:58.510669 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 60/123] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Oct 31 12:19:58 2022] GET /identity/v3/role_assignments?user.id=d2ddf15ba0c7488b8e49f974bba93de7&scope.project.id=9bc6131ae0634994bd48138d77878576&role.id=176c07b4fe3b4537bdfeb4be21b379c5 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:19:58.512063 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e92c1a5-cff6-4bb0-9e32-4439a0b13665 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:19:58.548831 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:19:58.584723 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1c01fd-fe7c-4246-8d45-acb4171d23c8 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:19:58.631710 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea0fe46-07a3-4c61-8d3d-7dfe5dda0370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:19:58.632589 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea0fe46-07a3-4c61-8d3d-7dfe5dda0370 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:58.633042 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea0fe46-07a3-4c61-8d3d-7dfe5dda0370 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:58.633464 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea0fe46-07a3-4c61-8d3d-7dfe5dda0370 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:58.689872 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:19:58.716640 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e92c1a5-cff6-4bb0-9e32-4439a0b13665 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:19:58.773307 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea0fe46-07a3-4c61-8d3d-7dfe5dda0370 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:19:58.774780 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea0fe46-07a3-4c61-8d3d-7dfe5dda0370 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:19:58.787503 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9172955-d6aa-417c-b83a-300f1026d195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:19:58.788387 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9172955-d6aa-417c-b83a-300f1026d195 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:58.788808 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9172955-d6aa-417c-b83a-300f1026d195 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:58.789218 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9172955-d6aa-417c-b83a-300f1026d195 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:58.853365 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:19:58.867918 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9172955-d6aa-417c-b83a-300f1026d195 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:19:58.869674 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9172955-d6aa-417c-b83a-300f1026d195 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:19:58.891284 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 64/124] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Oct 31 12:19:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 31 12:19:58.897345 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea0fe46-07a3-4c61-8d3d-7dfe5dda0370 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:19:58.966013 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:19:58.983311 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 61/125] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Oct 31 12:19:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 31 12:19:58.989126 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9172955-d6aa-417c-b83a-300f1026d195 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:19:59.005306 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea0fe46-07a3-4c61-8d3d-7dfe5dda0370 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:19:59.064514 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59246972-6049-462f-a228-202d7e1c353d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:19:59.065552 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59246972-6049-462f-a228-202d7e1c353d None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:59.066078 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59246972-6049-462f-a228-202d7e1c353d None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:59.066537 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59246972-6049-462f-a228-202d7e1c353d None admin] PATH_INFO: `/v3/projects/9bc6131ae0634994bd48138d77878576` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:59.067930 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59246972-6049-462f-a228-202d7e1c353d None admin] RBAC: Authorizing `identity:get_project(project_id=9bc6131ae0634994bd48138d77878576)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:19:59.069780 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59246972-6049-462f-a228-202d7e1c353d None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:19:59.071745 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 65/126] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Oct 31 12:19:58 2022] GET /identity/v3/projects/9bc6131ae0634994bd48138d77878576 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:19:59.083197 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59246972-6049-462f-a228-202d7e1c353d None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:19:59.100884 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9172955-d6aa-417c-b83a-300f1026d195 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:19:59.123660 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:19:59.137420 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8c9675-248a-4ee0-affb-3cb96ecf74cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:19:59.138066 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8c9675-248a-4ee0-affb-3cb96ecf74cd None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:59.138348 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8c9675-248a-4ee0-affb-3cb96ecf74cd None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:59.138629 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8c9675-248a-4ee0-affb-3cb96ecf74cd None admin] PATH_INFO: `/v3/projects/5cb588f8521c4eed8837380aa4af4156` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:59.139776 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8c9675-248a-4ee0-affb-3cb96ecf74cd None admin] RBAC: Authorizing `identity:get_project(project_id=5cb588f8521c4eed8837380aa4af4156)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:19:59.141349 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8c9675-248a-4ee0-affb-3cb96ecf74cd None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:19:59.143364 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 62/127] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Oct 31 12:19:58 2022] GET /identity/v3/projects/5cb588f8521c4eed8837380aa4af4156 => generated 312 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:19:59.149099 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8c9675-248a-4ee0-affb-3cb96ecf74cd None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:19:59.189104 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59246972-6049-462f-a228-202d7e1c353d None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:19:59.215402 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:19:59.236100 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7670fa2-2ffa-4f9e-9bc4-6f9a3fbb6a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:19:59.237132 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7670fa2-2ffa-4f9e-9bc4-6f9a3fbb6a36 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:59.237635 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7670fa2-2ffa-4f9e-9bc4-6f9a3fbb6a36 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:59.238164 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7670fa2-2ffa-4f9e-9bc4-6f9a3fbb6a36 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:59.239084 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7670fa2-2ffa-4f9e-9bc4-6f9a3fbb6a36 None admin] RBAC: Adding query filter params (role.id=65728d8bb0e546788426da322c34baba, scope.project.id=9bc6131ae0634994bd48138d77878576, user.id=d5915aed320e40a1ba7723089db18636) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:19:59.239613 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7670fa2-2ffa-4f9e-9bc4-6f9a3fbb6a36 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:19:59.241283 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7670fa2-2ffa-4f9e-9bc4-6f9a3fbb6a36 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:19:59.254339 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 66/128] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Oct 31 12:19:59 2022] GET /identity/v3/role_assignments?user.id=d5915aed320e40a1ba7723089db18636&scope.project.id=9bc6131ae0634994bd48138d77878576&role.id=65728d8bb0e546788426da322c34baba => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:19:59.275121 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8c9675-248a-4ee0-affb-3cb96ecf74cd None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:19:59.316467 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996034db-f2cb-47ef-a590-6582af6d09d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:19:59.317101 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996034db-f2cb-47ef-a590-6582af6d09d9 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:59.317382 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996034db-f2cb-47ef-a590-6582af6d09d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:59.317656 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996034db-f2cb-47ef-a590-6582af6d09d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:59.318404 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996034db-f2cb-47ef-a590-6582af6d09d9 None admin] RBAC: Adding query filter params (role.id=ca1a5695f8bd499aa0b8cedaeecd2cc6, scope.project.id=5cb588f8521c4eed8837380aa4af4156, user.id=d5915aed320e40a1ba7723089db18636) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:19:59.318714 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996034db-f2cb-47ef-a590-6582af6d09d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:19:59.320245 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996034db-f2cb-47ef-a590-6582af6d09d9 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:19:59.327374 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 63/129] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Oct 31 12:19:59 2022] GET /identity/v3/role_assignments?user.id=d5915aed320e40a1ba7723089db18636&scope.project.id=5cb588f8521c4eed8837380aa4af4156&role.id=ca1a5695f8bd499aa0b8cedaeecd2cc6 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 31 12:19:59.384116 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c5152a-8ea1-4d93-a246-60249cd24096 None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:59.384291 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c5152a-8ea1-4d93-a246-60249cd24096 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:59.384568 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c5152a-8ea1-4d93-a246-60249cd24096 None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:59.386949 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 67/130] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:19:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:19:59.393309 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed048114-6fb5-427a-bb90-95804fe50e30 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:59.393645 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed048114-6fb5-427a-bb90-95804fe50e30 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:59.393864 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed048114-6fb5-427a-bb90-95804fe50e30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:59.416185 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-ed048114-6fb5-427a-bb90-95804fe50e30 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:19:59.418816 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-ed048114-6fb5-427a-bb90-95804fe50e30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:19:59.504860 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 64/131] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:19:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 31 12:19:59.575721 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84ba5b7-7695-44a7-8ea9-dc0aafbc3ef4 None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:59.576025 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84ba5b7-7695-44a7-8ea9-dc0aafbc3ef4 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:59.576265 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84ba5b7-7695-44a7-8ea9-dc0aafbc3ef4 None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:59.577636 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 68/132] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:19:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:19:59.582846 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce1d507-1f26-44fa-90ec-0d06b18bef1a None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:59.583137 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce1d507-1f26-44fa-90ec-0d06b18bef1a None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:59.583447 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce1d507-1f26-44fa-90ec-0d06b18bef1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:59.601773 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-3ce1d507-1f26-44fa-90ec-0d06b18bef1a None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:19:59.602935 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-3ce1d507-1f26-44fa-90ec-0d06b18bef1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:19:59.645601 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:19:59.668292 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 65/133] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:19:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 31 12:19:59.776255 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84ba5b7-7695-44a7-8ea9-dc0aafbc3ef4 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:19:59.876141 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84ba5b7-7695-44a7-8ea9-dc0aafbc3ef4 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:19:59.878366 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:19:59.909669 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb212da-26e2-4ced-b23a-0ec10cfa377f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:19:59.910232 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb212da-26e2-4ced-b23a-0ec10cfa377f None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:19:59.910513 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb212da-26e2-4ced-b23a-0ec10cfa377f None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:19:59.910787 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb212da-26e2-4ced-b23a-0ec10cfa377f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:19:59.911899 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb212da-26e2-4ced-b23a-0ec10cfa377f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:19:59.913335 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb212da-26e2-4ced-b23a-0ec10cfa377f None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:19:59.914881 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 69/134] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Oct 31 12:19:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:19:59.921009 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce1d507-1f26-44fa-90ec-0d06b18bef1a None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:19:59.933667 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb212da-26e2-4ced-b23a-0ec10cfa377f None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:00.020398 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce1d507-1f26-44fa-90ec-0d06b18bef1a None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:00.036160 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb212da-26e2-4ced-b23a-0ec10cfa377f None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:00.058509 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c6446a-e44c-4ba1-9d41-9e987575e4be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:00.059070 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c6446a-e44c-4ba1-9d41-9e987575e4be None admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:00.059387 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c6446a-e44c-4ba1-9d41-9e987575e4be None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:00.059664 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c6446a-e44c-4ba1-9d41-9e987575e4be None admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:00.060317 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c6446a-e44c-4ba1-9d41-9e987575e4be None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:00.061748 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c6446a-e44c-4ba1-9d41-9e987575e4be None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:00.069259 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5da848-80dd-4c31-9f25-18aed84223be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:00.069883 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5da848-80dd-4c31-9f25-18aed84223be None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:00.070158 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5da848-80dd-4c31-9f25-18aed84223be None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:00.070435 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5da848-80dd-4c31-9f25-18aed84223be None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:00.081825 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 66/135] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Oct 31 12:19:59 2022] POST /identity/v3/users => generated 283 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:20:00.082001 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5da848-80dd-4c31-9f25-18aed84223be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 31 12:20:00.087537 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-ed5da848-80dd-4c31-9f25-18aed84223be None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 31 12:20:00.087537 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:00.087537 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:00.087537 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:00.087537 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:00.087537 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:00.087537 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:00.087537 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:00.087537 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:00.087537 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:00.087537 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:00.087537 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:00.087537 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 31 12:20:00.087537 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 31 12:20:00.087537 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 31 12:20:00.087537 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 31 12:20:00.089371 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 31 12:20:00.089371 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 31 12:20:00.089371 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 31 12:20:00.089371 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 31 12:20:00.089371 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:00.089371 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:00.089371 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:00.089371 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:00.089371 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:00.089371 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:00.089371 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:00.089371 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:00.089371 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:00.089371 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:00.089371 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:00.089371 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:00.090614 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:00.090614 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:00.090614 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:00.090614 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:00.090614 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 31 12:20:00.090614 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 31 12:20:00.090614 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 31 12:20:00.090614 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 31 12:20:00.090614 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 31 12:20:00.090614 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 31 12:20:00.090614 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 31 12:20:00.090614 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:20:00.090614 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 70/136] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Oct 31 12:19:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 31 12:20:00.096579 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c6446a-e44c-4ba1-9d41-9e987575e4be None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:00.132433 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:00.203041 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c6446a-e44c-4ba1-9d41-9e987575e4be None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:00.238495 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d95cf55-6a69-43bf-a536-d7060c523fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:00.239120 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d95cf55-6a69-43bf-a536-d7060c523fc5 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:00.239441 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d95cf55-6a69-43bf-a536-d7060c523fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:00.239718 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d95cf55-6a69-43bf-a536-d7060c523fc5 None admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:00.240389 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d95cf55-6a69-43bf-a536-d7060c523fc5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:00.240698 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d95cf55-6a69-43bf-a536-d7060c523fc5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:00.242069 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d95cf55-6a69-43bf-a536-d7060c523fc5 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:00.251437 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 67/137] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Oct 31 12:20:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:00.257268 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5da848-80dd-4c31-9f25-18aed84223be None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:00.290345 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:00.359882 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5da848-80dd-4c31-9f25-18aed84223be None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:00.394678 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08097d36-e97e-46be-a87f-105ed6191c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:00.395357 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08097d36-e97e-46be-a87f-105ed6191c4b None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:00.395639 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08097d36-e97e-46be-a87f-105ed6191c4b None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:00.395912 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08097d36-e97e-46be-a87f-105ed6191c4b None admin] PATH_INFO: `/v3/users/f1215134839e4b60a3fb995403b82a9a` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:00.397025 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08097d36-e97e-46be-a87f-105ed6191c4b None admin] RBAC: Authorizing `identity:get_user(user_id=f1215134839e4b60a3fb995403b82a9a)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:00.398498 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08097d36-e97e-46be-a87f-105ed6191c4b None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:00.400317 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 71/138] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Oct 31 12:20:00 2022] GET /identity/v3/users/f1215134839e4b60a3fb995403b82a9a => generated 289 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:00.406637 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d95cf55-6a69-43bf-a536-d7060c523fc5 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:00.507988 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:00.515316 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d95cf55-6a69-43bf-a536-d7060c523fc5 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:00.551887 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b5b8b0-7498-470a-9a96-87334b551d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:00.552503 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b5b8b0-7498-470a-9a96-87334b551d90 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:00.552779 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b5b8b0-7498-470a-9a96-87334b551d90 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:00.553048 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b5b8b0-7498-470a-9a96-87334b551d90 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:00.625980 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b5b8b0-7498-470a-9a96-87334b551d90 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:00.627577 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b5b8b0-7498-470a-9a96-87334b551d90 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:00.728007 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:00.736374 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 68/139] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Oct 31 12:20:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 31 12:20:00.741957 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08097d36-e97e-46be-a87f-105ed6191c4b None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:00.845326 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:00.848203 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08097d36-e97e-46be-a87f-105ed6191c4b None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:00.883266 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecee70e-e0e2-4d10-96ae-d058d2cdca7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:00.883905 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecee70e-e0e2-4d10-96ae-d058d2cdca7e None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:00.884172 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecee70e-e0e2-4d10-96ae-d058d2cdca7e None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:00.884449 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecee70e-e0e2-4d10-96ae-d058d2cdca7e None admin] PATH_INFO: `/v3/projects/9bc6131ae0634994bd48138d77878576` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:00.886241 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecee70e-e0e2-4d10-96ae-d058d2cdca7e None admin] RBAC: Authorizing `identity:get_project(project_id=9bc6131ae0634994bd48138d77878576)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:00.887708 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecee70e-e0e2-4d10-96ae-d058d2cdca7e None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:00.894609 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 72/140] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Oct 31 12:20:00 2022] GET /identity/v3/projects/9bc6131ae0634994bd48138d77878576 => generated 298 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:00.896021 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b5b8b0-7498-470a-9a96-87334b551d90 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:01.005403 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b5b8b0-7498-470a-9a96-87334b551d90 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:01.043809 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4db4980-e81a-47a0-a8e5-2c6315fc5570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:01.044423 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4db4980-e81a-47a0-a8e5-2c6315fc5570 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:01.044700 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4db4980-e81a-47a0-a8e5-2c6315fc5570 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:01.044968 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4db4980-e81a-47a0-a8e5-2c6315fc5570 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:01.045687 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4db4980-e81a-47a0-a8e5-2c6315fc5570 None admin] RBAC: Adding query filter params (role.id=056e0848b1da43f1a1eaae44d5ecddc8, scope.project.id=9bc6131ae0634994bd48138d77878576, user.id=f1215134839e4b60a3fb995403b82a9a) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:01.045994 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4db4980-e81a-47a0-a8e5-2c6315fc5570 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:01.047451 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4db4980-e81a-47a0-a8e5-2c6315fc5570 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:01.053915 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 69/141] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Oct 31 12:20:00 2022] GET /identity/v3/role_assignments?user.id=f1215134839e4b60a3fb995403b82a9a&scope.project.id=9bc6131ae0634994bd48138d77878576&role.id=056e0848b1da43f1a1eaae44d5ecddc8 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:02.373003 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134780c1-0846-46d0-9d09-1bba41cf4471 None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:02.373965 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134780c1-0846-46d0-9d09-1bba41cf4471 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:02.374461 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134780c1-0846-46d0-9d09-1bba41cf4471 None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:02.376370 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 73/142] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:20:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:20:02.381051 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979b492e-48d1-4ff9-83f4-a44980109cdd None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:02.381469 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979b492e-48d1-4ff9-83f4-a44980109cdd None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:02.381639 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979b492e-48d1-4ff9-83f4-a44980109cdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:02.401456 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-979b492e-48d1-4ff9-83f4-a44980109cdd None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:20:02.402766 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-979b492e-48d1-4ff9-83f4-a44980109cdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:20:02.464154 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:02.476443 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 70/143] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:20:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 31 12:20:02.522441 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54c986f-0ddf-49b7-b5c4-c91a71c643f2 None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:02.522728 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54c986f-0ddf-49b7-b5c4-c91a71c643f2 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:02.523022 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54c986f-0ddf-49b7-b5c4-c91a71c643f2 None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:02.524554 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 74/144] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:20:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:20:02.529610 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ec595b-fc40-4757-b6a7-ff3c2d1ee242 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:02.529899 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ec595b-fc40-4757-b6a7-ff3c2d1ee242 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:02.530177 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ec595b-fc40-4757-b6a7-ff3c2d1ee242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:02.549133 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-52ec595b-fc40-4757-b6a7-ff3c2d1ee242 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:20:02.550359 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-52ec595b-fc40-4757-b6a7-ff3c2d1ee242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:20:02.621543 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 71/145] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:20:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 31 12:20:02.675047 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cd9334-f8c7-445f-9c22-73faf0d54e1d None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:02.675313 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cd9334-f8c7-445f-9c22-73faf0d54e1d None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:02.675750 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cd9334-f8c7-445f-9c22-73faf0d54e1d None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:02.677309 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 75/146] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:20:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:20:02.683119 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0b0186-c1a6-44bf-90a3-93f56f0a6707 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:02.683433 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0b0186-c1a6-44bf-90a3-93f56f0a6707 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:02.683854 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0b0186-c1a6-44bf-90a3-93f56f0a6707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:02.702235 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-2c0b0186-c1a6-44bf-90a3-93f56f0a6707 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:20:02.703425 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-2c0b0186-c1a6-44bf-90a3-93f56f0a6707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:20:02.746592 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cd9334-f8c7-445f-9c22-73faf0d54e1d None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:02.774242 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 72/147] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:20:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 31 12:20:02.851768 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cd9334-f8c7-445f-9c22-73faf0d54e1d None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:02.886444 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c1cf34-f4d9-48db-8b9a-d5afdbe785d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:02.887368 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c1cf34-f4d9-48db-8b9a-d5afdbe785d2 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:02.887822 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c1cf34-f4d9-48db-8b9a-d5afdbe785d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:02.888285 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c1cf34-f4d9-48db-8b9a-d5afdbe785d2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:02.892972 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0b0186-c1a6-44bf-90a3-93f56f0a6707 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:02.899644 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c1cf34-f4d9-48db-8b9a-d5afdbe785d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 31 12:20:02.906216 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-35c1cf34-f4d9-48db-8b9a-d5afdbe785d2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 31 12:20:02.906216 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:02.906216 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:02.906216 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:02.906216 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:02.906216 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:02.906216 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:02.906216 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:02.906216 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:02.906216 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:02.906216 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:02.906216 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:02.906216 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 31 12:20:02.906216 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 31 12:20:02.906216 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 31 12:20:02.906216 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 31 12:20:02.907062 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 31 12:20:02.907062 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 31 12:20:02.907062 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 31 12:20:02.907062 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 31 12:20:02.907062 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:02.907062 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:02.907062 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:02.907062 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:02.907062 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:02.907062 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:02.907062 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:02.907062 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:02.907062 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:02.907062 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:02.907062 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:02.907062 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:02.907849 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:02.907849 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:02.907849 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:02.907849 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:02.907849 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 31 12:20:02.907849 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 31 12:20:02.907849 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 31 12:20:02.907849 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 31 12:20:02.907849 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 31 12:20:02.907849 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 31 12:20:02.907849 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 31 12:20:02.907849 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:20:02.911201 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 76/148] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Oct 31 12:20:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 31 12:20:02.914878 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c1cf34-f4d9-48db-8b9a-d5afdbe785d2 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:02.944344 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:02.945071 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:03.000974 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0b0186-c1a6-44bf-90a3-93f56f0a6707 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:03.020651 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c1cf34-f4d9-48db-8b9a-d5afdbe785d2 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:03.041665 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f749f259-eabe-4c31-a255-8cf7456aa168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:03.042776 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f749f259-eabe-4c31-a255-8cf7456aa168 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:03.043057 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f749f259-eabe-4c31-a255-8cf7456aa168 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:03.043358 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f749f259-eabe-4c31-a255-8cf7456aa168 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:03.054649 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f749f259-eabe-4c31-a255-8cf7456aa168 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 31 12:20:03.059474 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ebf508-f667-4d5c-9df1-b7948c2aff36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:03.060079 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ebf508-f667-4d5c-9df1-b7948c2aff36 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:03.060243 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-f749f259-eabe-4c31-a255-8cf7456aa168 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 31 12:20:03.060243 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:03.060243 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:03.060243 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:03.060243 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:03.060243 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:03.060243 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:03.060243 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:03.060243 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:03.060243 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:03.060243 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:03.060243 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:03.060243 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 31 12:20:03.060243 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 31 12:20:03.060243 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 31 12:20:03.060243 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 31 12:20:03.061184 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 31 12:20:03.061184 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 31 12:20:03.061184 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 31 12:20:03.061184 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 31 12:20:03.061184 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:03.061184 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:03.061184 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:03.061184 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:03.061184 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:03.061184 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:03.061184 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:03.061184 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:03.061184 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:03.061184 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:03.061184 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:03.061184 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:03.061958 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:03.061958 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:03.061958 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:03.061958 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:03.061958 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 31 12:20:03.061958 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 31 12:20:03.061958 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 31 12:20:03.061958 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 31 12:20:03.061958 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 31 12:20:03.061958 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 31 12:20:03.061958 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 31 12:20:03.061958 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:20:03.062557 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ebf508-f667-4d5c-9df1-b7948c2aff36 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:03.062557 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ebf508-f667-4d5c-9df1-b7948c2aff36 None admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:03.062557 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ebf508-f667-4d5c-9df1-b7948c2aff36 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:03.062818 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 73/149] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Oct 31 12:20:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 31 12:20:03.062885 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ebf508-f667-4d5c-9df1-b7948c2aff36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:03.062955 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f749f259-eabe-4c31-a255-8cf7456aa168 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:03.063024 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ebf508-f667-4d5c-9df1-b7948c2aff36 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:03.074016 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 77/150] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Oct 31 12:20:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 31 12:20:03.074349 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ebf508-f667-4d5c-9df1-b7948c2aff36 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:03.164386 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f749f259-eabe-4c31-a255-8cf7456aa168 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:03.164892 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:03.175559 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ebf508-f667-4d5c-9df1-b7948c2aff36 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:03.175837 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:03.201088 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bff2f0-70e9-493e-b261-473ec8f826a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:03.201652 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bff2f0-70e9-493e-b261-473ec8f826a5 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:03.201930 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bff2f0-70e9-493e-b261-473ec8f826a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:03.202198 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bff2f0-70e9-493e-b261-473ec8f826a5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:03.203328 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bff2f0-70e9-493e-b261-473ec8f826a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:03.204766 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bff2f0-70e9-493e-b261-473ec8f826a5 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:03.206316 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 74/151] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Oct 31 12:20:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:03.207079 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bff2f0-70e9-493e-b261-473ec8f826a5 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:03.209822 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c823ff-2924-4cf9-b27c-0358b48ca261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:03.210463 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c823ff-2924-4cf9-b27c-0358b48ca261 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:03.210748 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c823ff-2924-4cf9-b27c-0358b48ca261 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:03.211020 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c823ff-2924-4cf9-b27c-0358b48ca261 None admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:03.211726 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c823ff-2924-4cf9-b27c-0358b48ca261 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:03.212032 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c823ff-2924-4cf9-b27c-0358b48ca261 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:03.213998 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c823ff-2924-4cf9-b27c-0358b48ca261 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:03.222426 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 78/152] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Oct 31 12:20:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:03.223209 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c823ff-2924-4cf9-b27c-0358b48ca261 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:03.306430 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bff2f0-70e9-493e-b261-473ec8f826a5 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:03.319214 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c823ff-2924-4cf9-b27c-0358b48ca261 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:03.340242 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09db987f-12b3-4970-b733-a7268e270b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:03.340870 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09db987f-12b3-4970-b733-a7268e270b34 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:03.341148 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09db987f-12b3-4970-b733-a7268e270b34 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:03.341420 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09db987f-12b3-4970-b733-a7268e270b34 None admin] PATH_INFO: `/v3/users/c1b8e5ffbfb84d78ba40049ddd2017e8` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:03.352663 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cbb242-43dd-46a5-a5d2-1ee3ce7ce34f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:03.353260 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cbb242-43dd-46a5-a5d2-1ee3ce7ce34f None admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:03.353548 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cbb242-43dd-46a5-a5d2-1ee3ce7ce34f None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:03.354337 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cbb242-43dd-46a5-a5d2-1ee3ce7ce34f None admin] PATH_INFO: `/v3/users` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:03.354600 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09db987f-12b3-4970-b733-a7268e270b34 None admin] RBAC: Authorizing `identity:get_user(user_id=c1b8e5ffbfb84d78ba40049ddd2017e8)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:03.355080 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cbb242-43dd-46a5-a5d2-1ee3ce7ce34f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:03.356075 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09db987f-12b3-4970-b733-a7268e270b34 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:03.356521 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cbb242-43dd-46a5-a5d2-1ee3ce7ce34f None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:03.357672 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 75/153] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Oct 31 12:20:03 2022] GET /identity/v3/users/c1b8e5ffbfb84d78ba40049ddd2017e8 => generated 283 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:03.358446 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09db987f-12b3-4970-b733-a7268e270b34 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:03.387644 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 79/154] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Oct 31 12:20:03 2022] POST /identity/v3/users => generated 297 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:20:03.388515 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cbb242-43dd-46a5-a5d2-1ee3ce7ce34f None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:03.398818 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:03.400476 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:03.480508 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09db987f-12b3-4970-b733-a7268e270b34 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:03.506905 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cbb242-43dd-46a5-a5d2-1ee3ce7ce34f None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:03.523351 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7540e55b-30b3-45bf-acd4-aa07cb1d9e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:03.524053 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7540e55b-30b3-45bf-acd4-aa07cb1d9e6f None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:03.524339 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7540e55b-30b3-45bf-acd4-aa07cb1d9e6f None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:03.524617 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7540e55b-30b3-45bf-acd4-aa07cb1d9e6f None admin] PATH_INFO: `/v3/users/c1b8e5ffbfb84d78ba40049ddd2017e8` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:03.526352 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7540e55b-30b3-45bf-acd4-aa07cb1d9e6f None admin] RBAC: Authorizing `identity:get_user(user_id=c1b8e5ffbfb84d78ba40049ddd2017e8)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:03.527879 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7540e55b-30b3-45bf-acd4-aa07cb1d9e6f None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:03.529731 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 76/155] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Oct 31 12:20:03 2022] GET /identity/v3/users/c1b8e5ffbfb84d78ba40049ddd2017e8 => generated 283 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:03.535509 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7540e55b-30b3-45bf-acd4-aa07cb1d9e6f None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:03.545284 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669a51c3-b2ae-4082-9012-223259dbd47a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:03.545936 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669a51c3-b2ae-4082-9012-223259dbd47a None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:03.546211 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669a51c3-b2ae-4082-9012-223259dbd47a None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:03.546478 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669a51c3-b2ae-4082-9012-223259dbd47a None admin] PATH_INFO: `/v3/projects/574ce97a765a42729f79bf108a26330a` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:03.547670 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669a51c3-b2ae-4082-9012-223259dbd47a None admin] RBAC: Authorizing `identity:get_project(project_id=574ce97a765a42729f79bf108a26330a)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:03.549133 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669a51c3-b2ae-4082-9012-223259dbd47a None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:03.550787 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 80/156] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Oct 31 12:20:03 2022] GET /identity/v3/projects/574ce97a765a42729f79bf108a26330a => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:03.556603 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669a51c3-b2ae-4082-9012-223259dbd47a None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:03.641583 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:03.665538 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7540e55b-30b3-45bf-acd4-aa07cb1d9e6f None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:03.670361 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:03.671077 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669a51c3-b2ae-4082-9012-223259dbd47a None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:03.713834 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51646f0-3819-4119-a027-a99bf4919f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:03.714487 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51646f0-3819-4119-a027-a99bf4919f6e None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:03.714783 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51646f0-3819-4119-a027-a99bf4919f6e None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:03.715000 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51646f0-3819-4119-a027-a99bf4919f6e None admin] PATH_INFO: `/v3/projects/574ce97a765a42729f79bf108a26330a` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:03.716167 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51646f0-3819-4119-a027-a99bf4919f6e None admin] RBAC: Authorizing `identity:get_project(project_id=574ce97a765a42729f79bf108a26330a)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:03.717598 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51646f0-3819-4119-a027-a99bf4919f6e None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:03.719399 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 77/157] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Oct 31 12:20:03 2022] GET /identity/v3/projects/574ce97a765a42729f79bf108a26330a => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:03.727415 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1b07e2-cc22-438d-8be1-dea209281082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:03.727415 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1b07e2-cc22-438d-8be1-dea209281082 None admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:03.727415 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1b07e2-cc22-438d-8be1-dea209281082 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:03.727415 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1b07e2-cc22-438d-8be1-dea209281082 None admin] PATH_INFO: `/v3/projects/574ce97a765a42729f79bf108a26330a/users/c1b8e5ffbfb84d78ba40049ddd2017e8/roles/176c07b4fe3b4537bdfeb4be21b379c5` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:03.727777 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1b07e2-cc22-438d-8be1-dea209281082 None admin] RBAC: Authorizing `identity:create_grant(project_id=574ce97a765a42729f79bf108a26330a, user_id=c1b8e5ffbfb84d78ba40049ddd2017e8, role_id=176c07b4fe3b4537bdfeb4be21b379c5)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:03.728220 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1b07e2-cc22-438d-8be1-dea209281082 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:03.728320 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51646f0-3819-4119-a027-a99bf4919f6e None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:03.751695 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 81/158] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Oct 31 12:20:03 2022] PUT /identity/v3/projects/574ce97a765a42729f79bf108a26330a/users/c1b8e5ffbfb84d78ba40049ddd2017e8/roles/176c07b4fe3b4537bdfeb4be21b379c5 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 31 12:20:03.863023 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51646f0-3819-4119-a027-a99bf4919f6e None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:03.905014 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d5ffa7-ce72-42f9-97ae-71d1bdd9d790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:03.905694 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d5ffa7-ce72-42f9-97ae-71d1bdd9d790 None admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:03.905969 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d5ffa7-ce72-42f9-97ae-71d1bdd9d790 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:03.906309 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d5ffa7-ce72-42f9-97ae-71d1bdd9d790 None admin] PATH_INFO: `/v3/projects/574ce97a765a42729f79bf108a26330a/users/c1b8e5ffbfb84d78ba40049ddd2017e8/roles/65728d8bb0e546788426da322c34baba` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:03.908656 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d5ffa7-ce72-42f9-97ae-71d1bdd9d790 None admin] RBAC: Authorizing `identity:create_grant(project_id=574ce97a765a42729f79bf108a26330a, user_id=c1b8e5ffbfb84d78ba40049ddd2017e8, role_id=65728d8bb0e546788426da322c34baba)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:03.910166 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d5ffa7-ce72-42f9-97ae-71d1bdd9d790 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:03.930544 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 78/159] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Oct 31 12:20:03 2022] PUT /identity/v3/projects/574ce97a765a42729f79bf108a26330a/users/c1b8e5ffbfb84d78ba40049ddd2017e8/roles/65728d8bb0e546788426da322c34baba => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:20:05.693056 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e7a0e7-0fe3-4da6-a43f-c93b4fe0e730 None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:05.693570 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e7a0e7-0fe3-4da6-a43f-c93b4fe0e730 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:05.693674 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e7a0e7-0fe3-4da6-a43f-c93b4fe0e730 None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:05.695090 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 82/160] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:20:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:20:05.701738 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e28782e-76ca-42af-90d0-b0f9cdc0f77c None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:05.702742 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e28782e-76ca-42af-90d0-b0f9cdc0f77c None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:05.703255 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e28782e-76ca-42af-90d0-b0f9cdc0f77c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:05.726294 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-0e28782e-76ca-42af-90d0-b0f9cdc0f77c None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:20:05.727674 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-0e28782e-76ca-42af-90d0-b0f9cdc0f77c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:20:05.756778 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:05.804522 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 79/161] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:20:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 31 12:20:05.808030 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ff23f9-fe4c-4a5a-8191-38db06367ba7 None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:05.808471 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ff23f9-fe4c-4a5a-8191-38db06367ba7 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:05.808743 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ff23f9-fe4c-4a5a-8191-38db06367ba7 None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:05.810092 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 83/162] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:20:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:20:05.815272 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2926ce60-bee4-468d-b588-ad80e9db46c9 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:05.815562 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2926ce60-bee4-468d-b588-ad80e9db46c9 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:05.815837 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2926ce60-bee4-468d-b588-ad80e9db46c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:05.834932 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-2926ce60-bee4-468d-b588-ad80e9db46c9 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:20:05.836183 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-2926ce60-bee4-468d-b588-ad80e9db46c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:20:05.914427 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 80/163] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:20:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 31 12:20:05.968799 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e4cd98-1b19-4b18-971a-cfd0294b318c None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:05.969083 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e4cd98-1b19-4b18-971a-cfd0294b318c None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:05.969271 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e4cd98-1b19-4b18-971a-cfd0294b318c None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:05.970633 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 84/164] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:20:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:20:05.976067 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639dc45a-843c-43f6-9c40-3ae470341c79 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:05.976352 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639dc45a-843c-43f6-9c40-3ae470341c79 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:05.976668 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639dc45a-843c-43f6-9c40-3ae470341c79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:05.995746 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-639dc45a-843c-43f6-9c40-3ae470341c79 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:20:05.996926 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-639dc45a-843c-43f6-9c40-3ae470341c79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:20:06.080385 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 81/165] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:20:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 31 12:20:06.087108 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e4cd98-1b19-4b18-971a-cfd0294b318c None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:06.159877 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99737d17-b424-46b9-b513-9375fe1bb81c None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:06.160216 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99737d17-b424-46b9-b513-9375fe1bb81c None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:06.160421 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99737d17-b424-46b9-b513-9375fe1bb81c None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:06.162013 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 82/166] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:20:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 31 12:20:06.169296 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f927ee75-d487-4d7e-ad39-b0871624829e None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:06.169614 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f927ee75-d487-4d7e-ad39-b0871624829e None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:06.170003 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f927ee75-d487-4d7e-ad39-b0871624829e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:06.189363 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-f927ee75-d487-4d7e-ad39-b0871624829e None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:20:06.190772 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-f927ee75-d487-4d7e-ad39-b0871624829e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:20:06.197229 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e4cd98-1b19-4b18-971a-cfd0294b318c None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:06.209415 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:06.239984 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b702b43-af4c-4b79-b4b7-7ec7853260a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:06.240824 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b702b43-af4c-4b79-b4b7-7ec7853260a6 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:06.241108 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b702b43-af4c-4b79-b4b7-7ec7853260a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:06.241384 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b702b43-af4c-4b79-b4b7-7ec7853260a6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:06.245278 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:06.255689 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b702b43-af4c-4b79-b4b7-7ec7853260a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 31 12:20:06.262151 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-4b702b43-af4c-4b79-b4b7-7ec7853260a6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 31 12:20:06.262151 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:06.262151 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:06.262151 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:06.262151 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:06.262151 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:06.262151 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:06.262151 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:06.262151 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:06.262151 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:06.262151 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:06.262151 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:06.262151 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 31 12:20:06.262151 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 31 12:20:06.262151 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 31 12:20:06.262151 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 31 12:20:06.263082 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 31 12:20:06.263082 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 31 12:20:06.263082 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 31 12:20:06.263082 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 31 12:20:06.263082 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:06.263082 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:06.263082 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:06.263082 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:06.263082 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:06.263082 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:06.263082 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:06.263082 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:06.263082 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:06.263082 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:06.263082 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:06.263082 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:06.268688 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:06.268688 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:06.268688 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:06.268688 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:06.268688 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 31 12:20:06.268688 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 31 12:20:06.268688 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 31 12:20:06.268688 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 31 12:20:06.268688 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 31 12:20:06.268688 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 31 12:20:06.268688 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 31 12:20:06.268688 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:20:06.268688 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 85/167] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Oct 31 12:20:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:20:06.268688 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b702b43-af4c-4b79-b4b7-7ec7853260a6 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:06.272393 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 83/168] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:20:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 31 12:20:06.273470 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f927ee75-d487-4d7e-ad39-b0871624829e None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:06.388531 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b702b43-af4c-4b79-b4b7-7ec7853260a6 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:06.397338 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f927ee75-d487-4d7e-ad39-b0871624829e None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:06.428588 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300e5af5-53d0-48f6-adf4-9dde89412526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:06.429898 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300e5af5-53d0-48f6-adf4-9dde89412526 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:06.430505 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300e5af5-53d0-48f6-adf4-9dde89412526 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:06.431045 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300e5af5-53d0-48f6-adf4-9dde89412526 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:06.433066 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300e5af5-53d0-48f6-adf4-9dde89412526 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:06.434491 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300e5af5-53d0-48f6-adf4-9dde89412526 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:06.435534 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1daec9c-fb11-4a9b-8341-403dc4fe4dd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:06.439265 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 86/169] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Oct 31 12:20:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:06.439385 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1daec9c-fb11-4a9b-8341-403dc4fe4dd7 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:06.439385 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1daec9c-fb11-4a9b-8341-403dc4fe4dd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:06.439385 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1daec9c-fb11-4a9b-8341-403dc4fe4dd7 None admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:06.439562 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300e5af5-53d0-48f6-adf4-9dde89412526 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:06.439646 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1daec9c-fb11-4a9b-8341-403dc4fe4dd7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:06.439646 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1daec9c-fb11-4a9b-8341-403dc4fe4dd7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:06.439646 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1daec9c-fb11-4a9b-8341-403dc4fe4dd7 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:06.447234 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 84/170] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Oct 31 12:20:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:06.448395 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1daec9c-fb11-4a9b-8341-403dc4fe4dd7 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:06.469207 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:06.508654 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:06.549485 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300e5af5-53d0-48f6-adf4-9dde89412526 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:06.564570 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1daec9c-fb11-4a9b-8341-403dc4fe4dd7 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:06.583255 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5f8a30-404a-4262-9e35-e2fbd4413185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:06.583874 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5f8a30-404a-4262-9e35-e2fbd4413185 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:06.584149 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5f8a30-404a-4262-9e35-e2fbd4413185 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:06.584423 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5f8a30-404a-4262-9e35-e2fbd4413185 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:06.594947 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5f8a30-404a-4262-9e35-e2fbd4413185 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 31 12:20:06.598733 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7111c4b1-3c70-4e45-9b97-a1ba34ffbc34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:06.599365 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7111c4b1-3c70-4e45-9b97-a1ba34ffbc34 None admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:06.599645 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7111c4b1-3c70-4e45-9b97-a1ba34ffbc34 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:06.599860 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-db5f8a30-404a-4262-9e35-e2fbd4413185 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 31 12:20:06.599860 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:06.599860 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:06.599860 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:06.599860 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:06.599860 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:06.599860 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:06.599860 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:06.599860 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:06.599860 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:06.599860 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:06.599860 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:06.599860 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 31 12:20:06.599860 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 31 12:20:06.599860 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 31 12:20:06.599860 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 31 12:20:06.600861 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 31 12:20:06.600861 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 31 12:20:06.600861 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 31 12:20:06.600861 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 31 12:20:06.600861 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:06.600861 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:06.600861 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:06.600861 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:06.600861 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:06.600861 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:06.600861 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:06.600861 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:06.600861 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:06.600861 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:06.600861 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:06.600861 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:06.601629 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:06.601629 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:06.601629 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:06.601629 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:06.601629 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 31 12:20:06.601629 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 31 12:20:06.601629 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 31 12:20:06.601629 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 31 12:20:06.601629 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 31 12:20:06.601629 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 31 12:20:06.601629 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 31 12:20:06.601629 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:20:06.602406 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7111c4b1-3c70-4e45-9b97-a1ba34ffbc34 None admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:06.602406 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7111c4b1-3c70-4e45-9b97-a1ba34ffbc34 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:06.602529 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 87/171] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Oct 31 12:20:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:20:06.602529 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5f8a30-404a-4262-9e35-e2fbd4413185 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:06.602643 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7111c4b1-3c70-4e45-9b97-a1ba34ffbc34 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:06.631285 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 85/172] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Oct 31 12:20:06 2022] POST /identity/v3/users => generated 297 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:20:06.631285 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7111c4b1-3c70-4e45-9b97-a1ba34ffbc34 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:06.712141 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:06.722013 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5f8a30-404a-4262-9e35-e2fbd4413185 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:06.748863 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:06.753614 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7111c4b1-3c70-4e45-9b97-a1ba34ffbc34 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:06.767257 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0aefb0-a4b0-4503-a879-0b67bb17e436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:06.768103 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0aefb0-a4b0-4503-a879-0b67bb17e436 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:06.768382 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0aefb0-a4b0-4503-a879-0b67bb17e436 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:06.768650 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0aefb0-a4b0-4503-a879-0b67bb17e436 None admin] PATH_INFO: `/v3/users/6b9387f0deae4b32a3bfce455077a722` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:06.783705 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0aefb0-a4b0-4503-a879-0b67bb17e436 None admin] RBAC: Authorizing `identity:get_user(user_id=6b9387f0deae4b32a3bfce455077a722)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:06.785179 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0aefb0-a4b0-4503-a879-0b67bb17e436 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:06.791832 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 88/173] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Oct 31 12:20:06 2022] GET /identity/v3/users/6b9387f0deae4b32a3bfce455077a722 => generated 297 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:06.791832 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0aefb0-a4b0-4503-a879-0b67bb17e436 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:06.796833 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42491d78-1b02-4f84-9b4b-4d4e38d5c199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:06.797479 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42491d78-1b02-4f84-9b4b-4d4e38d5c199 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:06.797752 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42491d78-1b02-4f84-9b4b-4d4e38d5c199 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:06.798017 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42491d78-1b02-4f84-9b4b-4d4e38d5c199 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:06.812089 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42491d78-1b02-4f84-9b4b-4d4e38d5c199 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 31 12:20:06.817465 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-42491d78-1b02-4f84-9b4b-4d4e38d5c199 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 31 12:20:06.817465 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:06.817465 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:06.817465 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:06.817465 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:06.817465 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:06.817465 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:06.817465 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:06.817465 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:06.817465 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:06.817465 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:06.817465 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:06.817465 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 31 12:20:06.817465 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 31 12:20:06.817465 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 31 12:20:06.817465 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 31 12:20:06.818353 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 31 12:20:06.818353 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 31 12:20:06.818353 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 31 12:20:06.818353 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 31 12:20:06.818353 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:06.818353 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:06.818353 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:06.818353 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:06.818353 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:06.818353 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:06.818353 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:06.818353 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:06.818353 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:06.818353 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:06.818353 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:06.818353 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:06.819134 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:06.819134 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:06.819134 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:06.819134 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:06.819134 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 31 12:20:06.819134 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 31 12:20:06.819134 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 31 12:20:06.819134 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 31 12:20:06.819134 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 31 12:20:06.819134 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 31 12:20:06.819134 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 31 12:20:06.819134 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:20:06.820745 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 86/174] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Oct 31 12:20:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:20:06.820745 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42491d78-1b02-4f84-9b4b-4d4e38d5c199 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:06.921531 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0aefb0-a4b0-4503-a879-0b67bb17e436 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:06.940969 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42491d78-1b02-4f84-9b4b-4d4e38d5c199 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:06.962598 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2202a2b8-d326-4618-80f7-c71160b3ca4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:06.963303 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2202a2b8-d326-4618-80f7-c71160b3ca4b None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:06.963650 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2202a2b8-d326-4618-80f7-c71160b3ca4b None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:06.963926 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2202a2b8-d326-4618-80f7-c71160b3ca4b None admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:06.964740 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2202a2b8-d326-4618-80f7-c71160b3ca4b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:06.965042 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2202a2b8-d326-4618-80f7-c71160b3ca4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:06.966953 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2202a2b8-d326-4618-80f7-c71160b3ca4b None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:06.975235 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:06.976562 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 89/175] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Oct 31 12:20:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:06.977202 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2202a2b8-d326-4618-80f7-c71160b3ca4b None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:06.985698 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c253a0f-2ed8-41a5-8969-19ccc36aa1a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:06.986650 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c253a0f-2ed8-41a5-8969-19ccc36aa1a3 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:06.987209 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c253a0f-2ed8-41a5-8969-19ccc36aa1a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:06.987695 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c253a0f-2ed8-41a5-8969-19ccc36aa1a3 None admin] PATH_INFO: `/v3/projects/574ce97a765a42729f79bf108a26330a` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:06.989046 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c253a0f-2ed8-41a5-8969-19ccc36aa1a3 None admin] RBAC: Authorizing `identity:get_project(project_id=574ce97a765a42729f79bf108a26330a)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:06.990877 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c253a0f-2ed8-41a5-8969-19ccc36aa1a3 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:06.993142 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 87/176] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Oct 31 12:20:06 2022] GET /identity/v3/projects/574ce97a765a42729f79bf108a26330a => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:06.994159 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c253a0f-2ed8-41a5-8969-19ccc36aa1a3 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:07.034143 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:07.105185 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2202a2b8-d326-4618-80f7-c71160b3ca4b None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:07.128112 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c253a0f-2ed8-41a5-8969-19ccc36aa1a3 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:07.149356 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b951c444-02a7-4912-951b-30af305554be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:07.149975 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b951c444-02a7-4912-951b-30af305554be None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:07.150254 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b951c444-02a7-4912-951b-30af305554be None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:07.150526 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b951c444-02a7-4912-951b-30af305554be None admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:07.151918 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b951c444-02a7-4912-951b-30af305554be None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:07.152221 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b951c444-02a7-4912-951b-30af305554be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:07.154134 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b951c444-02a7-4912-951b-30af305554be None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:07.165548 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 90/177] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Oct 31 12:20:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:07.166911 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b951c444-02a7-4912-951b-30af305554be None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:07.167760 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7759c3b-da43-47ec-8415-cf8a76f83e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:07.168488 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7759c3b-da43-47ec-8415-cf8a76f83e3e None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:07.168829 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7759c3b-da43-47ec-8415-cf8a76f83e3e None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:07.169166 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7759c3b-da43-47ec-8415-cf8a76f83e3e None admin] PATH_INFO: `/v3/users/c1b8e5ffbfb84d78ba40049ddd2017e8` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:07.170380 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7759c3b-da43-47ec-8415-cf8a76f83e3e None admin] RBAC: Authorizing `identity:get_user(user_id=c1b8e5ffbfb84d78ba40049ddd2017e8)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:07.171995 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7759c3b-da43-47ec-8415-cf8a76f83e3e None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:07.173983 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 88/178] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Oct 31 12:20:06 2022] GET /identity/v3/users/c1b8e5ffbfb84d78ba40049ddd2017e8 => generated 283 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:07.174883 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7759c3b-da43-47ec-8415-cf8a76f83e3e None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:07.249138 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:07.293333 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7759c3b-da43-47ec-8415-cf8a76f83e3e None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:07.298990 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b951c444-02a7-4912-951b-30af305554be None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:07.306663 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:07.334708 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2b11dc-75c1-4730-9cab-beb1a162d6b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:07.335647 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2b11dc-75c1-4730-9cab-beb1a162d6b7 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:07.335925 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2b11dc-75c1-4730-9cab-beb1a162d6b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:07.336196 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2b11dc-75c1-4730-9cab-beb1a162d6b7 None admin] PATH_INFO: `/v3/users/c1b8e5ffbfb84d78ba40049ddd2017e8` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:07.337316 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2b11dc-75c1-4730-9cab-beb1a162d6b7 None admin] RBAC: Authorizing `identity:get_user(user_id=c1b8e5ffbfb84d78ba40049ddd2017e8)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:07.339678 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2b11dc-75c1-4730-9cab-beb1a162d6b7 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:07.339869 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3fba71-199e-477c-a454-8ed36e20abda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:07.340556 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3fba71-199e-477c-a454-8ed36e20abda None admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:07.340894 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3fba71-199e-477c-a454-8ed36e20abda None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:07.341225 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3fba71-199e-477c-a454-8ed36e20abda None admin] PATH_INFO: `/v3/projects/574ce97a765a42729f79bf108a26330a/users/6b9387f0deae4b32a3bfce455077a722/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:07.341586 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 89/179] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Oct 31 12:20:07 2022] GET /identity/v3/users/c1b8e5ffbfb84d78ba40049ddd2017e8 => generated 283 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:07.342363 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2b11dc-75c1-4730-9cab-beb1a162d6b7 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:07.343702 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3fba71-199e-477c-a454-8ed36e20abda None admin] RBAC: Authorizing `identity:create_grant(project_id=574ce97a765a42729f79bf108a26330a, user_id=6b9387f0deae4b32a3bfce455077a722, role_id=ca1a5695f8bd499aa0b8cedaeecd2cc6)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:07.345395 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3fba71-199e-477c-a454-8ed36e20abda None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:07.360843 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 91/180] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Oct 31 12:20:07 2022] PUT /identity/v3/projects/574ce97a765a42729f79bf108a26330a/users/6b9387f0deae4b32a3bfce455077a722/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:20:07.368224 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3fba71-199e-477c-a454-8ed36e20abda None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:07.529669 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2b11dc-75c1-4730-9cab-beb1a162d6b7 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:07.583042 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3fba71-199e-477c-a454-8ed36e20abda None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:07.600890 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9052efe9-6efe-4549-b995-88d48ba003a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:07.601501 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9052efe9-6efe-4549-b995-88d48ba003a9 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:07.601775 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9052efe9-6efe-4549-b995-88d48ba003a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:07.602050 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9052efe9-6efe-4549-b995-88d48ba003a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:07.606483 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:07.639817 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7026380a-a617-4a15-af34-d690d33dca1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:07.640654 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7026380a-a617-4a15-af34-d690d33dca1b None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:07.641420 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7026380a-a617-4a15-af34-d690d33dca1b None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:07.641708 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7026380a-a617-4a15-af34-d690d33dca1b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:07.688252 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:07.703658 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9052efe9-6efe-4549-b995-88d48ba003a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:07.705404 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9052efe9-6efe-4549-b995-88d48ba003a9 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:07.743587 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7026380a-a617-4a15-af34-d690d33dca1b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:07.745252 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7026380a-a617-4a15-af34-d690d33dca1b None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:07.842029 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 90/181] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Oct 31 12:20:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 31 12:20:07.849274 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9052efe9-6efe-4549-b995-88d48ba003a9 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:07.906705 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:07.953679 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 92/182] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Oct 31 12:20:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 592 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 31 12:20:07.959442 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7026380a-a617-4a15-af34-d690d33dca1b None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:07.986417 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9052efe9-6efe-4549-b995-88d48ba003a9 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:08.030580 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a921746-b268-48ce-acd7-b8f48a965ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:08.032511 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a921746-b268-48ce-acd7-b8f48a965ff5 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:08.032511 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a921746-b268-48ce-acd7-b8f48a965ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:08.032511 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a921746-b268-48ce-acd7-b8f48a965ff5 None admin] PATH_INFO: `/v3/projects/574ce97a765a42729f79bf108a26330a` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:08.033543 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a921746-b268-48ce-acd7-b8f48a965ff5 None admin] RBAC: Authorizing `identity:get_project(project_id=574ce97a765a42729f79bf108a26330a)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:08.035185 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a921746-b268-48ce-acd7-b8f48a965ff5 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:08.036888 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 91/183] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Oct 31 12:20:07 2022] GET /identity/v3/projects/574ce97a765a42729f79bf108a26330a => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 31 12:20:08.042947 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a921746-b268-48ce-acd7-b8f48a965ff5 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:08.149561 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a921746-b268-48ce-acd7-b8f48a965ff5 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:08.152206 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:08.182617 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:08.187688 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149be084-a872-4a79-8e70-2fa9211ca0c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:08.188525 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149be084-a872-4a79-8e70-2fa9211ca0c8 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:08.188978 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149be084-a872-4a79-8e70-2fa9211ca0c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:08.189446 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149be084-a872-4a79-8e70-2fa9211ca0c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:08.190341 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149be084-a872-4a79-8e70-2fa9211ca0c8 None admin] RBAC: Adding query filter params (role.id=176c07b4fe3b4537bdfeb4be21b379c5, scope.project.id=574ce97a765a42729f79bf108a26330a, user.id=c1b8e5ffbfb84d78ba40049ddd2017e8) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:08.190821 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149be084-a872-4a79-8e70-2fa9211ca0c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:08.192447 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149be084-a872-4a79-8e70-2fa9211ca0c8 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:08.199149 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 92/184] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Oct 31 12:20:08 2022] GET /identity/v3/role_assignments?user.id=c1b8e5ffbfb84d78ba40049ddd2017e8&scope.project.id=574ce97a765a42729f79bf108a26330a&role.id=176c07b4fe3b4537bdfeb4be21b379c5 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 31 12:20:08.267430 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7026380a-a617-4a15-af34-d690d33dca1b None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:08.321480 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48a1414-e347-4ecb-83d5-c2cf245bd89b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:08.322125 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48a1414-e347-4ecb-83d5-c2cf245bd89b None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:08.322403 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48a1414-e347-4ecb-83d5-c2cf245bd89b None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:08.323308 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48a1414-e347-4ecb-83d5-c2cf245bd89b None admin] PATH_INFO: `/v3/projects/574ce97a765a42729f79bf108a26330a` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:08.328074 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48a1414-e347-4ecb-83d5-c2cf245bd89b None admin] RBAC: Authorizing `identity:get_project(project_id=574ce97a765a42729f79bf108a26330a)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:08.329557 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48a1414-e347-4ecb-83d5-c2cf245bd89b None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:08.331311 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 93/185] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Oct 31 12:20:07 2022] GET /identity/v3/projects/574ce97a765a42729f79bf108a26330a => generated 302 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:08.337325 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149be084-a872-4a79-8e70-2fa9211ca0c8 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:08.494109 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149be084-a872-4a79-8e70-2fa9211ca0c8 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:08.543034 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6630945-eaaa-4cc6-a4d5-737a482934bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:08.543701 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6630945-eaaa-4cc6-a4d5-737a482934bc None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:08.543978 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6630945-eaaa-4cc6-a4d5-737a482934bc None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:08.544242 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6630945-eaaa-4cc6-a4d5-737a482934bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:08.544966 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6630945-eaaa-4cc6-a4d5-737a482934bc None admin] RBAC: Adding query filter params (role.id=65728d8bb0e546788426da322c34baba, scope.project.id=574ce97a765a42729f79bf108a26330a, user.id=c1b8e5ffbfb84d78ba40049ddd2017e8) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:08.545269 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6630945-eaaa-4cc6-a4d5-737a482934bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:08.546682 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6630945-eaaa-4cc6-a4d5-737a482934bc None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:08.554737 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 93/186] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Oct 31 12:20:08 2022] GET /identity/v3/role_assignments?user.id=c1b8e5ffbfb84d78ba40049ddd2017e8&scope.project.id=574ce97a765a42729f79bf108a26330a&role.id=65728d8bb0e546788426da322c34baba => generated 608 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:09.072049 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fcdead-e135-4867-8847-828ced4c5070 None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:09.072609 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fcdead-e135-4867-8847-828ced4c5070 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:09.072609 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fcdead-e135-4867-8847-828ced4c5070 None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:09.074459 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 94/187] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:20:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:20:09.079678 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfe511c-c8c3-420b-ba3d-858d799d4a1d None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:09.080075 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfe511c-c8c3-420b-ba3d-858d799d4a1d None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:09.080256 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfe511c-c8c3-420b-ba3d-858d799d4a1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:09.098774 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-2bfe511c-c8c3-420b-ba3d-858d799d4a1d None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:20:09.100000 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-2bfe511c-c8c3-420b-ba3d-858d799d4a1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:20:09.124362 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:09.174027 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 94/188] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:20:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 31 12:20:09.261818 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b85eba-0a53-45f8-9716-33e6b14bb2da None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:09.262103 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b85eba-0a53-45f8-9716-33e6b14bb2da None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:09.262552 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b85eba-0a53-45f8-9716-33e6b14bb2da None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:09.264079 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 95/189] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:20:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:20:09.269350 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073641d1-8e06-446a-8d86-1310e082e27a None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:09.269640 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073641d1-8e06-446a-8d86-1310e082e27a None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:09.269918 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073641d1-8e06-446a-8d86-1310e082e27a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:09.288615 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-073641d1-8e06-446a-8d86-1310e082e27a None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:20:09.289719 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-073641d1-8e06-446a-8d86-1310e082e27a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:20:09.360010 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 95/190] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:20:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 31 12:20:09.444315 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b85eba-0a53-45f8-9716-33e6b14bb2da None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:09.548602 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b85eba-0a53-45f8-9716-33e6b14bb2da None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:09.582495 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b48566e-3c58-4130-a59e-d024aa3cff55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:09.583184 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b48566e-3c58-4130-a59e-d024aa3cff55 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:09.583474 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b48566e-3c58-4130-a59e-d024aa3cff55 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:09.583742 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b48566e-3c58-4130-a59e-d024aa3cff55 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:09.585455 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:09.596737 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b48566e-3c58-4130-a59e-d024aa3cff55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 31 12:20:09.601606 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-0b48566e-3c58-4130-a59e-d024aa3cff55 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 31 12:20:09.601606 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:09.601606 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:09.601606 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:09.601606 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:09.601606 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:09.601606 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:09.601606 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:09.601606 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:09.601606 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:09.601606 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:09.601606 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:09.601606 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 31 12:20:09.601606 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 31 12:20:09.601606 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 31 12:20:09.601606 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 31 12:20:09.602478 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 31 12:20:09.602478 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 31 12:20:09.602478 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 31 12:20:09.602478 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 31 12:20:09.602478 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:09.602478 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:09.602478 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:09.602478 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:09.602478 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:09.602478 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:09.602478 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:09.602478 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:09.602478 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:09.602478 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:09.602478 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:09.602478 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:09.608136 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:09.608136 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:09.608136 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:09.608136 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:09.608136 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 31 12:20:09.608136 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 31 12:20:09.608136 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 31 12:20:09.608136 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 31 12:20:09.608136 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 31 12:20:09.608136 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 31 12:20:09.608136 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 31 12:20:09.608136 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:20:09.608136 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 96/191] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Oct 31 12:20:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:20:09.610397 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073641d1-8e06-446a-8d86-1310e082e27a None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:09.642150 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b48566e-3c58-4130-a59e-d024aa3cff55 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:09.715399 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:09.728430 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073641d1-8e06-446a-8d86-1310e082e27a None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:09.755268 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b48566e-3c58-4130-a59e-d024aa3cff55 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:09.766843 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed716a56-88f2-4a1b-9fbf-25eadd160c84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:09.767969 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed716a56-88f2-4a1b-9fbf-25eadd160c84 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:09.768233 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed716a56-88f2-4a1b-9fbf-25eadd160c84 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:09.768505 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed716a56-88f2-4a1b-9fbf-25eadd160c84 None admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:09.769213 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed716a56-88f2-4a1b-9fbf-25eadd160c84 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:09.769515 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed716a56-88f2-4a1b-9fbf-25eadd160c84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:09.770926 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed716a56-88f2-4a1b-9fbf-25eadd160c84 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:09.783748 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 96/192] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Oct 31 12:20:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:09.784810 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed716a56-88f2-4a1b-9fbf-25eadd160c84 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:09.798295 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43860b1c-7aae-48c8-93fe-e49180bdddb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:09.798976 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43860b1c-7aae-48c8-93fe-e49180bdddb6 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:09.799217 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43860b1c-7aae-48c8-93fe-e49180bdddb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:09.799487 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43860b1c-7aae-48c8-93fe-e49180bdddb6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:09.800575 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43860b1c-7aae-48c8-93fe-e49180bdddb6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:09.802002 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43860b1c-7aae-48c8-93fe-e49180bdddb6 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:09.804025 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 97/193] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Oct 31 12:20:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 31 12:20:09.809435 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43860b1c-7aae-48c8-93fe-e49180bdddb6 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:09.869021 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:09.889361 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed716a56-88f2-4a1b-9fbf-25eadd160c84 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:09.906840 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43860b1c-7aae-48c8-93fe-e49180bdddb6 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:09.922208 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b2cfb6-c3ab-4631-83d0-7d3a7384ff4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:09.922837 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b2cfb6-c3ab-4631-83d0-7d3a7384ff4a None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:09.923115 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b2cfb6-c3ab-4631-83d0-7d3a7384ff4a None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:09.923415 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b2cfb6-c3ab-4631-83d0-7d3a7384ff4a None admin] PATH_INFO: `/v3/users/ffa935410734468f91e0f126417c00d9` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:09.936367 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b2cfb6-c3ab-4631-83d0-7d3a7384ff4a None admin] RBAC: Authorizing `identity:get_user(user_id=ffa935410734468f91e0f126417c00d9)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:09.937801 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b2cfb6-c3ab-4631-83d0-7d3a7384ff4a None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:09.939428 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 97/194] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Oct 31 12:20:09 2022] GET /identity/v3/users/ffa935410734468f91e0f126417c00d9 => generated 297 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 31 12:20:09.940568 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9446a0c6-ff63-4b95-b57d-59e173343192 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:09.940849 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9446a0c6-ff63-4b95-b57d-59e173343192 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:09.941121 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9446a0c6-ff63-4b95-b57d-59e173343192 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:09.942309 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 98/195] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:20:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:20:09.944786 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfce99b-c2ba-422d-99db-61ae80cfe655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:09.945138 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9446a0c6-ff63-4b95-b57d-59e173343192 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:09.945391 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfce99b-c2ba-422d-99db-61ae80cfe655 None admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:09.945732 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfce99b-c2ba-422d-99db-61ae80cfe655 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:09.946005 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfce99b-c2ba-422d-99db-61ae80cfe655 None admin] PATH_INFO: `/v3/users` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:09.946727 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfce99b-c2ba-422d-99db-61ae80cfe655 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:09.948969 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfce99b-c2ba-422d-99db-61ae80cfe655 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:09.957644 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:09.969990 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 98/196] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Oct 31 12:20:09 2022] POST /identity/v3/users => generated 293 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 31 12:20:09.972499 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777e4438-322b-4f7e-a5cb-56815a991035 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:09.972791 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777e4438-322b-4f7e-a5cb-56815a991035 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:09.973097 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777e4438-322b-4f7e-a5cb-56815a991035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:09.993890 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-777e4438-322b-4f7e-a5cb-56815a991035 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:20:09.996222 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-777e4438-322b-4f7e-a5cb-56815a991035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:20:10.072152 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 99/197] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:20:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 31 12:20:10.078406 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9446a0c6-ff63-4b95-b57d-59e173343192 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:10.117061 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c8817d-f2a5-4d47-b322-01050776d868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:10.118412 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c8817d-f2a5-4d47-b322-01050776d868 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:10.118834 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c8817d-f2a5-4d47-b322-01050776d868 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:10.119294 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c8817d-f2a5-4d47-b322-01050776d868 None admin] PATH_INFO: `/v3/projects/574ce97a765a42729f79bf108a26330a` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:10.121371 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c8817d-f2a5-4d47-b322-01050776d868 None admin] RBAC: Authorizing `identity:get_project(project_id=574ce97a765a42729f79bf108a26330a)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:10.122973 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c8817d-f2a5-4d47-b322-01050776d868 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:10.124848 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 99/198] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Oct 31 12:20:09 2022] GET /identity/v3/projects/574ce97a765a42729f79bf108a26330a => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 31 12:20:10.131073 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777e4438-322b-4f7e-a5cb-56815a991035 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:10.187915 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:10.242425 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777e4438-322b-4f7e-a5cb-56815a991035 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:10.279343 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3cc75e-5935-4d0e-b5f2-8ae633836bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:10.280018 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3cc75e-5935-4d0e-b5f2-8ae633836bbe None admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:10.280296 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3cc75e-5935-4d0e-b5f2-8ae633836bbe None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:10.280568 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3cc75e-5935-4d0e-b5f2-8ae633836bbe None admin] PATH_INFO: `/v3/projects/574ce97a765a42729f79bf108a26330a/users/ffa935410734468f91e0f126417c00d9/roles/056e0848b1da43f1a1eaae44d5ecddc8` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:10.282480 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3cc75e-5935-4d0e-b5f2-8ae633836bbe None admin] RBAC: Authorizing `identity:create_grant(project_id=574ce97a765a42729f79bf108a26330a, user_id=ffa935410734468f91e0f126417c00d9, role_id=056e0848b1da43f1a1eaae44d5ecddc8)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:10.283963 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3cc75e-5935-4d0e-b5f2-8ae633836bbe None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:10.301455 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 100/199] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Oct 31 12:20:10 2022] PUT /identity/v3/projects/574ce97a765a42729f79bf108a26330a/users/ffa935410734468f91e0f126417c00d9/roles/056e0848b1da43f1a1eaae44d5ecddc8 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 31 12:20:10.351416 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c8817d-f2a5-4d47-b322-01050776d868 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:10.444910 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:10.469953 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c8817d-f2a5-4d47-b322-01050776d868 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:10.512810 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adc62cc-9246-4697-a1f2-6b20040d9599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:10.513447 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adc62cc-9246-4697-a1f2-6b20040d9599 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:10.513721 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adc62cc-9246-4697-a1f2-6b20040d9599 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:10.513986 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adc62cc-9246-4697-a1f2-6b20040d9599 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:10.526399 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adc62cc-9246-4697-a1f2-6b20040d9599 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 31 12:20:10.531524 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-2adc62cc-9246-4697-a1f2-6b20040d9599 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 31 12:20:10.531524 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:10.531524 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:10.531524 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:10.531524 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:10.531524 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:10.531524 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:10.531524 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:10.531524 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:10.531524 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:10.531524 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:10.531524 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:10.531524 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 31 12:20:10.531524 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 31 12:20:10.531524 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 31 12:20:10.531524 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 31 12:20:10.532417 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 31 12:20:10.532417 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 31 12:20:10.532417 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 31 12:20:10.532417 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 31 12:20:10.532417 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:10.532417 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:10.532417 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:10.532417 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:10.532417 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:10.532417 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:10.532417 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:10.532417 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:10.532417 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:10.532417 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:10.532417 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:10.532417 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:10.533207 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:10.533207 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:10.533207 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:10.533207 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:10.533207 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 31 12:20:10.533207 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 31 12:20:10.533207 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 31 12:20:10.533207 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 31 12:20:10.533207 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 31 12:20:10.533207 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 31 12:20:10.533207 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 31 12:20:10.533207 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:20:10.533207 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 100/200] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Oct 31 12:20:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:20:10.540209 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3cc75e-5935-4d0e-b5f2-8ae633836bbe None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:10.650512 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3cc75e-5935-4d0e-b5f2-8ae633836bbe None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:10.660913 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:10.689292 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3429ff5-89c7-4ad7-ba7e-3b03a17f72d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:10.689910 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3429ff5-89c7-4ad7-ba7e-3b03a17f72d0 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:10.690246 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3429ff5-89c7-4ad7-ba7e-3b03a17f72d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:10.690517 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3429ff5-89c7-4ad7-ba7e-3b03a17f72d0 None admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:10.691257 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3429ff5-89c7-4ad7-ba7e-3b03a17f72d0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:10.691562 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3429ff5-89c7-4ad7-ba7e-3b03a17f72d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:10.692919 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3429ff5-89c7-4ad7-ba7e-3b03a17f72d0 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:10.702651 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 101/201] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Oct 31 12:20:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:10.708325 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adc62cc-9246-4697-a1f2-6b20040d9599 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:10.818173 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adc62cc-9246-4697-a1f2-6b20040d9599 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:10.857751 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f89b57-237a-4186-b198-f89e82f029ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:10.858770 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f89b57-237a-4186-b198-f89e82f029ef None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:10.859267 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f89b57-237a-4186-b198-f89e82f029ef None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:10.859727 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f89b57-237a-4186-b198-f89e82f029ef None admin] PATH_INFO: `/v3/users/6b9387f0deae4b32a3bfce455077a722` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:10.861036 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f89b57-237a-4186-b198-f89e82f029ef None admin] RBAC: Authorizing `identity:get_user(user_id=6b9387f0deae4b32a3bfce455077a722)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:10.862655 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f89b57-237a-4186-b198-f89e82f029ef None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:10.864467 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 101/202] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Oct 31 12:20:10 2022] GET /identity/v3/users/6b9387f0deae4b32a3bfce455077a722 => generated 297 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:10.871210 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3429ff5-89c7-4ad7-ba7e-3b03a17f72d0 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:10.978530 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3429ff5-89c7-4ad7-ba7e-3b03a17f72d0 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:11.013409 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d49701-498c-4218-ae18-5a7b966d7394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:11.014024 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d49701-498c-4218-ae18-5a7b966d7394 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:11.014305 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d49701-498c-4218-ae18-5a7b966d7394 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:11.014575 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d49701-498c-4218-ae18-5a7b966d7394 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:11.068613 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:11.101644 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d49701-498c-4218-ae18-5a7b966d7394 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:11.103116 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d49701-498c-4218-ae18-5a7b966d7394 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:11.205345 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 102/203] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Oct 31 12:20:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 31 12:20:11.210650 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f89b57-237a-4186-b198-f89e82f029ef None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:11.227878 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:11.317762 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f89b57-237a-4186-b198-f89e82f029ef None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:11.357059 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5810cee1-0c82-4324-92aa-e7b1f106a393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:11.358029 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5810cee1-0c82-4324-92aa-e7b1f106a393 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:11.358496 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5810cee1-0c82-4324-92aa-e7b1f106a393 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:11.358969 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5810cee1-0c82-4324-92aa-e7b1f106a393 None admin] PATH_INFO: `/v3/projects/574ce97a765a42729f79bf108a26330a` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:11.360298 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5810cee1-0c82-4324-92aa-e7b1f106a393 None admin] RBAC: Authorizing `identity:get_project(project_id=574ce97a765a42729f79bf108a26330a)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:11.361894 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5810cee1-0c82-4324-92aa-e7b1f106a393 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:11.364096 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 102/204] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Oct 31 12:20:11 2022] GET /identity/v3/projects/574ce97a765a42729f79bf108a26330a => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:11.369637 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d49701-498c-4218-ae18-5a7b966d7394 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:11.445243 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:11.477813 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d49701-498c-4218-ae18-5a7b966d7394 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:11.514373 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c2a7f4-5834-4333-8072-c43d6b44faf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:11.515032 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c2a7f4-5834-4333-8072-c43d6b44faf8 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:11.515340 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c2a7f4-5834-4333-8072-c43d6b44faf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:11.515609 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c2a7f4-5834-4333-8072-c43d6b44faf8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:11.516324 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c2a7f4-5834-4333-8072-c43d6b44faf8 None admin] RBAC: Adding query filter params (role.id=ca1a5695f8bd499aa0b8cedaeecd2cc6, scope.project.id=574ce97a765a42729f79bf108a26330a, user.id=6b9387f0deae4b32a3bfce455077a722) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:11.516625 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c2a7f4-5834-4333-8072-c43d6b44faf8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:11.518314 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c2a7f4-5834-4333-8072-c43d6b44faf8 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:11.524395 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 103/205] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Oct 31 12:20:11 2022] GET /identity/v3/role_assignments?user.id=6b9387f0deae4b32a3bfce455077a722&scope.project.id=574ce97a765a42729f79bf108a26330a&role.id=ca1a5695f8bd499aa0b8cedaeecd2cc6 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:12.377889 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe92e78-4888-4895-ba7d-cdb74d9ce221 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:12.378368 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe92e78-4888-4895-ba7d-cdb74d9ce221 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:12.378368 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe92e78-4888-4895-ba7d-cdb74d9ce221 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:12.379751 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 103/206] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:20:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:20:12.384868 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a560b8-00ef-431a-8281-dde2d33be564 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:12.386210 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a560b8-00ef-431a-8281-dde2d33be564 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:12.386490 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a560b8-00ef-431a-8281-dde2d33be564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:12.404770 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-d4a560b8-00ef-431a-8281-dde2d33be564 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:20:12.405960 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-d4a560b8-00ef-431a-8281-dde2d33be564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:20:12.408241 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877c687b-05f5-48da-9616-14fc5366dcca None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:12.408525 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877c687b-05f5-48da-9616-14fc5366dcca None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:12.408797 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877c687b-05f5-48da-9616-14fc5366dcca None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:12.410123 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 104/207] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:20:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 31 12:20:12.415194 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26630ab0-82ff-4cfd-8ca1-dd612e22c6ee None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:12.415627 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26630ab0-82ff-4cfd-8ca1-dd612e22c6ee None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:12.415893 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26630ab0-82ff-4cfd-8ca1-dd612e22c6ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:12.434315 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-26630ab0-82ff-4cfd-8ca1-dd612e22c6ee None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:20:12.435513 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-26630ab0-82ff-4cfd-8ca1-dd612e22c6ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:20:12.479266 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 104/208] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:20:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 31 12:20:12.508596 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 105/209] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:20:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 31 12:20:12.587400 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd059f85-d4db-4809-b0d1-065660709c22 None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:12.587698 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd059f85-d4db-4809-b0d1-065660709c22 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:12.587948 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd059f85-d4db-4809-b0d1-065660709c22 None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:12.590172 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 105/210] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:20:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:20:12.595348 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ba0a61-fc31-477f-8132-184effd451ad None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:12.595643 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ba0a61-fc31-477f-8132-184effd451ad None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:12.595934 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ba0a61-fc31-477f-8132-184effd451ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:12.603083 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:12.615396 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-77ba0a61-fc31-477f-8132-184effd451ad None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:20:12.616654 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-77ba0a61-fc31-477f-8132-184effd451ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:20:12.685412 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 106/211] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:20:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 31 12:20:12.748805 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd059f85-d4db-4809-b0d1-065660709c22 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:12.774937 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ba0a61-fc31-477f-8132-184effd451ad None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:12.812756 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:12.853340 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd059f85-d4db-4809-b0d1-065660709c22 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:12.882162 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ba0a61-fc31-477f-8132-184effd451ad None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:12.889260 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bda243-c8ca-4bf8-845c-1e262502e7cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:12.889883 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bda243-c8ca-4bf8-845c-1e262502e7cc None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:12.890154 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bda243-c8ca-4bf8-845c-1e262502e7cc None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:12.890412 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bda243-c8ca-4bf8-845c-1e262502e7cc None admin] PATH_INFO: `/v3/roles/member` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:12.901795 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bda243-c8ca-4bf8-845c-1e262502e7cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 31 12:20:12.906682 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-c7bda243-c8ca-4bf8-845c-1e262502e7cc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 31 12:20:12.906682 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:12.906682 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:12.906682 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:12.906682 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:12.906682 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:12.906682 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:12.906682 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:12.906682 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:12.906682 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:12.906682 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:12.906682 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:12.906682 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 31 12:20:12.906682 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 31 12:20:12.906682 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 31 12:20:12.906682 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 31 12:20:12.907587 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 31 12:20:12.907587 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 31 12:20:12.907587 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 31 12:20:12.907587 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 31 12:20:12.907587 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:12.907587 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:12.907587 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:12.907587 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:12.907587 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:12.907587 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:12.907587 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:12.907587 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:12.907587 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:12.907587 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:12.907587 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:12.907587 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:12.908414 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:12.908414 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:12.908414 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:12.908414 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:12.908414 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 31 12:20:12.908414 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 31 12:20:12.908414 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 31 12:20:12.908414 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 31 12:20:12.908414 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 31 12:20:12.908414 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 31 12:20:12.908414 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 31 12:20:12.908414 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:20:12.908414 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 106/212] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Oct 31 12:20:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:20:12.913478 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bda243-c8ca-4bf8-845c-1e262502e7cc None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:12.921903 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd71e12-1e9f-4326-9d50-efc6fe123a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:12.922813 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd71e12-1e9f-4326-9d50-efc6fe123a4c None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:12.923090 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd71e12-1e9f-4326-9d50-efc6fe123a4c None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:12.923386 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd71e12-1e9f-4326-9d50-efc6fe123a4c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:12.924750 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd71e12-1e9f-4326-9d50-efc6fe123a4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:12.925879 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd71e12-1e9f-4326-9d50-efc6fe123a4c None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:12.931341 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 107/213] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Oct 31 12:20:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 31 12:20:12.933203 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd71e12-1e9f-4326-9d50-efc6fe123a4c None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:12.936271 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:13.017927 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bda243-c8ca-4bf8-845c-1e262502e7cc None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:13.038265 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd71e12-1e9f-4326-9d50-efc6fe123a4c None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:13.042063 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:13.052934 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8817f852-04d3-4469-a0ff-a92c7bd4a97d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:13.053541 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8817f852-04d3-4469-a0ff-a92c7bd4a97d None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:13.053817 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8817f852-04d3-4469-a0ff-a92c7bd4a97d None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:13.054084 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8817f852-04d3-4469-a0ff-a92c7bd4a97d None admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:13.054734 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8817f852-04d3-4469-a0ff-a92c7bd4a97d None admin] RBAC: Adding query filter params (name=member) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:13.055032 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8817f852-04d3-4469-a0ff-a92c7bd4a97d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:13.056420 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8817f852-04d3-4469-a0ff-a92c7bd4a97d None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:13.064532 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 107/214] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Oct 31 12:20:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 31 12:20:13.065309 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8817f852-04d3-4469-a0ff-a92c7bd4a97d None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:13.078398 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe95de1-e8c2-47c4-8c77-1e9b7e14ca9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:13.079009 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe95de1-e8c2-47c4-8c77-1e9b7e14ca9a None admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:13.079316 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe95de1-e8c2-47c4-8c77-1e9b7e14ca9a None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:13.079583 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe95de1-e8c2-47c4-8c77-1e9b7e14ca9a None admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:13.080239 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe95de1-e8c2-47c4-8c77-1e9b7e14ca9a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:13.081650 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe95de1-e8c2-47c4-8c77-1e9b7e14ca9a None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:13.102098 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 108/215] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Oct 31 12:20:12 2022] POST /identity/v3/users => generated 293 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:20:13.102977 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe95de1-e8c2-47c4-8c77-1e9b7e14ca9a None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:13.175501 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8817f852-04d3-4469-a0ff-a92c7bd4a97d None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:13.193537 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:13.214403 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e43020a-0a46-47ad-ac9b-a407b024708e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:13.215048 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e43020a-0a46-47ad-ac9b-a407b024708e None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:13.215370 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e43020a-0a46-47ad-ac9b-a407b024708e None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:13.215637 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e43020a-0a46-47ad-ac9b-a407b024708e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:13.228668 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e43020a-0a46-47ad-ac9b-a407b024708e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 31 12:20:13.234219 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-4e43020a-0a46-47ad-ac9b-a407b024708e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 31 12:20:13.234219 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:13.234219 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:13.234219 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:13.234219 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:13.234219 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:13.234219 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:13.234219 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:13.234219 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:13.234219 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:13.234219 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:13.234219 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:13.234219 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 31 12:20:13.234219 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 31 12:20:13.234219 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 31 12:20:13.234219 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 31 12:20:13.235241 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 31 12:20:13.235241 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 31 12:20:13.235241 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 31 12:20:13.235241 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 31 12:20:13.235241 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:13.235241 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:13.235241 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:13.235241 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:13.235241 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:13.235241 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:13.235241 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:13.235241 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:13.235241 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:13.235241 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:13.235241 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:13.235241 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:13.236034 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:13.236034 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:13.236034 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:13.236034 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:13.236034 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 31 12:20:13.236034 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 31 12:20:13.236034 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 31 12:20:13.236034 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 31 12:20:13.236034 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 31 12:20:13.236034 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 31 12:20:13.236034 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 31 12:20:13.236034 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:20:13.236034 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 108/216] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Oct 31 12:20:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:20:13.241162 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e43020a-0a46-47ad-ac9b-a407b024708e None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:13.249536 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe95de1-e8c2-47c4-8c77-1e9b7e14ca9a None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:13.290422 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9072a396-98d5-4ef2-b332-d89c198ab492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:13.291221 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9072a396-98d5-4ef2-b332-d89c198ab492 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:13.291567 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9072a396-98d5-4ef2-b332-d89c198ab492 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:13.291839 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9072a396-98d5-4ef2-b332-d89c198ab492 None admin] PATH_INFO: `/v3/users/77cb971b62a249188000632a460b7f5a` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:13.299004 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:13.310141 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9072a396-98d5-4ef2-b332-d89c198ab492 None admin] RBAC: Authorizing `identity:get_user(user_id=77cb971b62a249188000632a460b7f5a)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:13.311682 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9072a396-98d5-4ef2-b332-d89c198ab492 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:13.313526 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 109/217] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Oct 31 12:20:13 2022] GET /identity/v3/users/77cb971b62a249188000632a460b7f5a => generated 293 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:13.318963 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9072a396-98d5-4ef2-b332-d89c198ab492 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:13.360254 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e43020a-0a46-47ad-ac9b-a407b024708e None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:13.406746 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ba7946-b519-4aef-a0c8-8fc567bc5458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:13.406746 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ba7946-b519-4aef-a0c8-8fc567bc5458 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:13.406746 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ba7946-b519-4aef-a0c8-8fc567bc5458 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:13.406746 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ba7946-b519-4aef-a0c8-8fc567bc5458 None admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:13.407576 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ba7946-b519-4aef-a0c8-8fc567bc5458 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:13.407863 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ba7946-b519-4aef-a0c8-8fc567bc5458 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:13.410442 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ba7946-b519-4aef-a0c8-8fc567bc5458 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:13.418503 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 109/218] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Oct 31 12:20:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:13.424014 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ba7946-b519-4aef-a0c8-8fc567bc5458 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:13.443785 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9072a396-98d5-4ef2-b332-d89c198ab492 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:13.467572 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:13.490757 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc824436-b8bd-4588-8d33-df764251d8e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:13.491403 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc824436-b8bd-4588-8d33-df764251d8e3 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:13.491683 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc824436-b8bd-4588-8d33-df764251d8e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:13.491952 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc824436-b8bd-4588-8d33-df764251d8e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:13.492672 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc824436-b8bd-4588-8d33-df764251d8e3 None admin] RBAC: Adding query filter params (role.id=ca1a5695f8bd499aa0b8cedaeecd2cc6, user.id=77cb971b62a249188000632a460b7f5a, scope.system=all) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:13.492973 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc824436-b8bd-4588-8d33-df764251d8e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:13.494975 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc824436-b8bd-4588-8d33-df764251d8e3 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:13.501499 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 110/219] 10.222.0.15 () {56 vars in 1308 bytes} [Mon Oct 31 12:20:13 2022] GET /identity/v3/role_assignments?user.id=77cb971b62a249188000632a460b7f5a&scope.system=all&role.id=ca1a5695f8bd499aa0b8cedaeecd2cc6 => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 31 12:20:13.543585 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ba7946-b519-4aef-a0c8-8fc567bc5458 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:13.553903 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:13.582999 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9ddf02-76ad-4c31-9d8a-22e7499aa629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:13.583684 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9ddf02-76ad-4c31-9d8a-22e7499aa629 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:13.583961 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9ddf02-76ad-4c31-9d8a-22e7499aa629 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:13.584227 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9ddf02-76ad-4c31-9d8a-22e7499aa629 None admin] PATH_INFO: `/v3/users/ffa935410734468f91e0f126417c00d9` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:13.585459 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9ddf02-76ad-4c31-9d8a-22e7499aa629 None admin] RBAC: Authorizing `identity:get_user(user_id=ffa935410734468f91e0f126417c00d9)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:13.586918 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9ddf02-76ad-4c31-9d8a-22e7499aa629 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:13.589494 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 110/220] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Oct 31 12:20:13 2022] GET /identity/v3/users/ffa935410734468f91e0f126417c00d9 => generated 297 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:13.596338 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc824436-b8bd-4588-8d33-df764251d8e3 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:13.707464 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc824436-b8bd-4588-8d33-df764251d8e3 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:13.744735 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253ae1e6-b1ee-4fa1-9d5c-b47d338866e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:13.745347 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253ae1e6-b1ee-4fa1-9d5c-b47d338866e3 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:13.745624 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253ae1e6-b1ee-4fa1-9d5c-b47d338866e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:13.745894 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253ae1e6-b1ee-4fa1-9d5c-b47d338866e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:13.798795 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:13.820073 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253ae1e6-b1ee-4fa1-9d5c-b47d338866e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:13.821606 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253ae1e6-b1ee-4fa1-9d5c-b47d338866e3 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:13.929231 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 111/221] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Oct 31 12:20:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 31 12:20:13.936359 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9ddf02-76ad-4c31-9d8a-22e7499aa629 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:14.047821 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9ddf02-76ad-4c31-9d8a-22e7499aa629 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:14.082281 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef03aa29-eb11-47a9-9b6f-121bd44aa48d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:14.082958 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef03aa29-eb11-47a9-9b6f-121bd44aa48d None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:14.083272 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef03aa29-eb11-47a9-9b6f-121bd44aa48d None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:14.083549 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef03aa29-eb11-47a9-9b6f-121bd44aa48d None admin] PATH_INFO: `/v3/projects/574ce97a765a42729f79bf108a26330a` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:14.084723 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef03aa29-eb11-47a9-9b6f-121bd44aa48d None admin] RBAC: Authorizing `identity:get_project(project_id=574ce97a765a42729f79bf108a26330a)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:14.086247 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef03aa29-eb11-47a9-9b6f-121bd44aa48d None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:14.088884 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 111/222] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Oct 31 12:20:13 2022] GET /identity/v3/projects/574ce97a765a42729f79bf108a26330a => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:14.094925 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253ae1e6-b1ee-4fa1-9d5c-b47d338866e3 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:14.176969 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:14.198561 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253ae1e6-b1ee-4fa1-9d5c-b47d338866e3 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:14.234794 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdf82f3-8372-4e88-aa24-926f858230cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:14.235428 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdf82f3-8372-4e88-aa24-926f858230cf None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:14.235701 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdf82f3-8372-4e88-aa24-926f858230cf None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:14.236017 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdf82f3-8372-4e88-aa24-926f858230cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:14.236736 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdf82f3-8372-4e88-aa24-926f858230cf None admin] RBAC: Adding query filter params (role.id=056e0848b1da43f1a1eaae44d5ecddc8, scope.project.id=574ce97a765a42729f79bf108a26330a, user.id=ffa935410734468f91e0f126417c00d9) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:14.237375 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdf82f3-8372-4e88-aa24-926f858230cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:14.238794 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdf82f3-8372-4e88-aa24-926f858230cf None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:14.245382 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 112/223] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Oct 31 12:20:14 2022] GET /identity/v3/role_assignments?user.id=ffa935410734468f91e0f126417c00d9&scope.project.id=574ce97a765a42729f79bf108a26330a&role.id=056e0848b1da43f1a1eaae44d5ecddc8 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:15.486701 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db41c3d1-a256-46d8-a408-58a14645a25a None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:15.487235 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db41c3d1-a256-46d8-a408-58a14645a25a None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:15.487235 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db41c3d1-a256-46d8-a408-58a14645a25a None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:15.488540 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 112/224] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:20:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:20:15.493642 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae7ec99-e1b6-4d1b-9978-cce711ea341b None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:15.494032 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae7ec99-e1b6-4d1b-9978-cce711ea341b None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:15.494201 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae7ec99-e1b6-4d1b-9978-cce711ea341b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:15.512923 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-5ae7ec99-e1b6-4d1b-9978-cce711ea341b None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:20:15.514091 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-5ae7ec99-e1b6-4d1b-9978-cce711ea341b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:20:15.559693 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e275fa46-e571-4f04-8ab7-d79449f5d6ee None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:15.559970 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e275fa46-e571-4f04-8ab7-d79449f5d6ee None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:15.560245 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e275fa46-e571-4f04-8ab7-d79449f5d6ee None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:15.561599 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 113/225] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:20:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 31 12:20:15.566726 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7ec093-d6af-4dbc-b724-d16e3d107ef7 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:15.567013 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7ec093-d6af-4dbc-b724-d16e3d107ef7 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:15.567325 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7ec093-d6af-4dbc-b724-d16e3d107ef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:15.585630 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-5f7ec093-d6af-4dbc-b724-d16e3d107ef7 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:20:15.587380 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 113/226] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:20:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 31 12:20:15.587477 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-5f7ec093-d6af-4dbc-b724-d16e3d107ef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:20:15.632314 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:15.660446 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 114/227] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:20:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 31 12:20:15.707134 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0a4fe0-e6e7-4051-822d-14b45d31bd4d None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:15.707468 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0a4fe0-e6e7-4051-822d-14b45d31bd4d None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:15.707713 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0a4fe0-e6e7-4051-822d-14b45d31bd4d None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:15.709182 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 114/228] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:20:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:20:15.714861 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2309cb50-71fb-4234-9692-cebe1a564ea1 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:15.715146 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2309cb50-71fb-4234-9692-cebe1a564ea1 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:15.715443 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2309cb50-71fb-4234-9692-cebe1a564ea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:15.733318 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-2309cb50-71fb-4234-9692-cebe1a564ea1 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:20:15.734473 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-2309cb50-71fb-4234-9692-cebe1a564ea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:20:15.799929 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 115/229] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:20:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 31 12:20:15.861038 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0a4fe0-e6e7-4051-822d-14b45d31bd4d None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:15.935262 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2309cb50-71fb-4234-9692-cebe1a564ea1 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:15.936846 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:15.967151 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0a4fe0-e6e7-4051-822d-14b45d31bd4d None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:16.004254 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0618673e-1574-41ed-adf5-bb348a113088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:16.004886 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0618673e-1574-41ed-adf5-bb348a113088 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:16.005148 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0618673e-1574-41ed-adf5-bb348a113088 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:16.005415 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0618673e-1574-41ed-adf5-bb348a113088 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:16.017939 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0618673e-1574-41ed-adf5-bb348a113088 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 31 12:20:16.023102 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-0618673e-1574-41ed-adf5-bb348a113088 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 31 12:20:16.023102 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:16.023102 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:16.023102 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:16.023102 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:16.023102 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:16.023102 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:16.023102 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:16.023102 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:16.023102 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:16.023102 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:16.023102 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:16.023102 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 31 12:20:16.023102 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 31 12:20:16.023102 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 31 12:20:16.023102 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 31 12:20:16.024029 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 31 12:20:16.024029 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 31 12:20:16.024029 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 31 12:20:16.024029 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 31 12:20:16.024029 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:16.024029 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:16.024029 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:16.024029 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:16.024029 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:16.024029 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:16.024029 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:16.024029 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:16.024029 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:16.024029 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:16.024029 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:16.024029 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:16.029891 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:16.029891 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:16.029891 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:16.029891 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:16.029891 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 31 12:20:16.029891 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 31 12:20:16.029891 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 31 12:20:16.029891 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 31 12:20:16.029891 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 31 12:20:16.029891 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 31 12:20:16.029891 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 31 12:20:16.029891 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:20:16.029891 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 115/230] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Oct 31 12:20:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:20:16.031241 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0618673e-1574-41ed-adf5-bb348a113088 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:16.052082 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2309cb50-71fb-4234-9692-cebe1a564ea1 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:16.054374 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:16.087705 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716435f1-bb6d-4ab1-88d2-15c7caa9bf9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:16.088384 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716435f1-bb6d-4ab1-88d2-15c7caa9bf9e None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:16.088657 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716435f1-bb6d-4ab1-88d2-15c7caa9bf9e None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:16.088924 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716435f1-bb6d-4ab1-88d2-15c7caa9bf9e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:16.089987 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716435f1-bb6d-4ab1-88d2-15c7caa9bf9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:16.091448 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716435f1-bb6d-4ab1-88d2-15c7caa9bf9e None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:16.092993 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 116/231] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Oct 31 12:20:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 31 12:20:16.093733 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716435f1-bb6d-4ab1-88d2-15c7caa9bf9e None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:16.136870 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0618673e-1574-41ed-adf5-bb348a113088 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:16.170711 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad6737d-e341-4d04-a633-dc7587febb94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:16.171347 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad6737d-e341-4d04-a633-dc7587febb94 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:16.171622 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad6737d-e341-4d04-a633-dc7587febb94 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:16.171892 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad6737d-e341-4d04-a633-dc7587febb94 None admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:16.172566 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad6737d-e341-4d04-a633-dc7587febb94 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:16.172862 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad6737d-e341-4d04-a633-dc7587febb94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:16.174226 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad6737d-e341-4d04-a633-dc7587febb94 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:16.175657 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:16.184312 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 116/232] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Oct 31 12:20:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:16.185168 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad6737d-e341-4d04-a633-dc7587febb94 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:16.191102 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716435f1-bb6d-4ab1-88d2-15c7caa9bf9e None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:16.224415 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00091375-eb4f-4d74-a61d-d06b37505ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:16.225035 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00091375-eb4f-4d74-a61d-d06b37505ae5 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:16.225311 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00091375-eb4f-4d74-a61d-d06b37505ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:16.225578 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00091375-eb4f-4d74-a61d-d06b37505ae5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:16.235523 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00091375-eb4f-4d74-a61d-d06b37505ae5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 31 12:20:16.240350 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-00091375-eb4f-4d74-a61d-d06b37505ae5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 31 12:20:16.240350 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:16.240350 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:16.240350 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:16.240350 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:16.240350 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:16.240350 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:16.240350 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:16.240350 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:16.240350 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:16.240350 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:16.240350 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:16.240350 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 31 12:20:16.240350 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 31 12:20:16.240350 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 31 12:20:16.240350 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 31 12:20:16.241221 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 31 12:20:16.241221 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 31 12:20:16.241221 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 31 12:20:16.241221 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 31 12:20:16.241221 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:16.241221 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:16.241221 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:16.241221 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:16.241221 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:16.241221 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:16.241221 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:16.241221 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:16.241221 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:16.241221 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:16.241221 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:16.241221 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:16.241999 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:16.241999 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:16.241999 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:16.241999 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:16.241999 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 31 12:20:16.241999 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 31 12:20:16.241999 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 31 12:20:16.241999 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 31 12:20:16.241999 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 31 12:20:16.241999 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 31 12:20:16.241999 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 31 12:20:16.241999 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:20:16.241999 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 117/233] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Oct 31 12:20:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:20:16.242647 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00091375-eb4f-4d74-a61d-d06b37505ae5 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:16.272563 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:16.287933 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad6737d-e341-4d04-a633-dc7587febb94 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:16.321037 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9658dd-6eaf-4bdf-9669-66f0c1eac665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:16.321573 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9658dd-6eaf-4bdf-9669-66f0c1eac665 None admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:16.321848 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9658dd-6eaf-4bdf-9669-66f0c1eac665 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:16.322115 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9658dd-6eaf-4bdf-9669-66f0c1eac665 None admin] PATH_INFO: `/v3/groups` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:16.323628 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9658dd-6eaf-4bdf-9669-66f0c1eac665 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:16.324893 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9658dd-6eaf-4bdf-9669-66f0c1eac665 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:16.342108 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 117/234] 10.222.0.15 () {60 vars in 1134 bytes} [Mon Oct 31 12:20:16 2022] POST /identity/v3/groups => generated 229 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:20:16.342921 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9658dd-6eaf-4bdf-9669-66f0c1eac665 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:16.358101 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00091375-eb4f-4d74-a61d-d06b37505ae5 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:16.399776 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e7c19c-b485-4105-a0d4-7bc536363813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:16.400426 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e7c19c-b485-4105-a0d4-7bc536363813 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:16.400805 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e7c19c-b485-4105-a0d4-7bc536363813 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:16.401235 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e7c19c-b485-4105-a0d4-7bc536363813 None admin] PATH_INFO: `/v3/users/eb235f50b9714414a5ba4e523068829b` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:16.410707 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:16.417918 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e7c19c-b485-4105-a0d4-7bc536363813 None admin] RBAC: Authorizing `identity:get_user(user_id=eb235f50b9714414a5ba4e523068829b)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:16.420125 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e7c19c-b485-4105-a0d4-7bc536363813 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:16.426668 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 118/235] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Oct 31 12:20:16 2022] GET /identity/v3/users/eb235f50b9714414a5ba4e523068829b => generated 293 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:16.427881 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e7c19c-b485-4105-a0d4-7bc536363813 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:16.467463 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9658dd-6eaf-4bdf-9669-66f0c1eac665 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:16.504871 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ca6d2e-c1ce-4cad-bb89-20e7979d1e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:16.505525 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ca6d2e-c1ce-4cad-bb89-20e7979d1e8d None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:16.505756 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ca6d2e-c1ce-4cad-bb89-20e7979d1e8d None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:16.506027 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ca6d2e-c1ce-4cad-bb89-20e7979d1e8d None admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:16.506700 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ca6d2e-c1ce-4cad-bb89-20e7979d1e8d None admin] RBAC: Adding query filter params (name=member) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:16.506995 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ca6d2e-c1ce-4cad-bb89-20e7979d1e8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:16.508404 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ca6d2e-c1ce-4cad-bb89-20e7979d1e8d None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:16.517383 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 118/236] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Oct 31 12:20:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:16.523981 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ca6d2e-c1ce-4cad-bb89-20e7979d1e8d None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:16.544587 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:16.559304 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e7c19c-b485-4105-a0d4-7bc536363813 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:16.599609 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89b1aa6-8887-49aa-9c16-b3d934787611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:16.600156 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89b1aa6-8887-49aa-9c16-b3d934787611 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:16.600432 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89b1aa6-8887-49aa-9c16-b3d934787611 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:16.600701 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89b1aa6-8887-49aa-9c16-b3d934787611 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:16.601409 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89b1aa6-8887-49aa-9c16-b3d934787611 None admin] RBAC: Adding query filter params (role.id=056e0848b1da43f1a1eaae44d5ecddc8, user.id=eb235f50b9714414a5ba4e523068829b, scope.system=all) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:16.601711 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89b1aa6-8887-49aa-9c16-b3d934787611 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:16.603151 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89b1aa6-8887-49aa-9c16-b3d934787611 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:16.608216 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 119/237] 10.222.0.15 () {56 vars in 1308 bytes} [Mon Oct 31 12:20:16 2022] GET /identity/v3/role_assignments?user.id=eb235f50b9714414a5ba4e523068829b&scope.system=all&role.id=056e0848b1da43f1a1eaae44d5ecddc8 => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:16.637462 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ca6d2e-c1ce-4cad-bb89-20e7979d1e8d None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:16.657745 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:16.679186 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0ec0b5-6ae8-44d8-b3af-b6807d57605a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:16.679856 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0ec0b5-6ae8-44d8-b3af-b6807d57605a None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:16.680131 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0ec0b5-6ae8-44d8-b3af-b6807d57605a None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:16.680399 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0ec0b5-6ae8-44d8-b3af-b6807d57605a None admin] PATH_INFO: `/v3/users/77cb971b62a249188000632a460b7f5a` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:16.681667 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0ec0b5-6ae8-44d8-b3af-b6807d57605a None admin] RBAC: Authorizing `identity:get_user(user_id=77cb971b62a249188000632a460b7f5a)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:16.683327 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0ec0b5-6ae8-44d8-b3af-b6807d57605a None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:16.685182 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 119/238] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Oct 31 12:20:16 2022] GET /identity/v3/users/77cb971b62a249188000632a460b7f5a => generated 293 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:16.690763 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89b1aa6-8887-49aa-9c16-b3d934787611 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:16.808000 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89b1aa6-8887-49aa-9c16-b3d934787611 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:16.842574 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fb5c75-30d3-4391-acb6-88571c1f22a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:16.843245 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fb5c75-30d3-4391-acb6-88571c1f22a7 None admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:16.843524 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fb5c75-30d3-4391-acb6-88571c1f22a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:16.843791 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fb5c75-30d3-4391-acb6-88571c1f22a7 None admin] PATH_INFO: `/v3/system/users/77cb971b62a249188000632a460b7f5a/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:16.845171 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fb5c75-30d3-4391-acb6-88571c1f22a7 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=77cb971b62a249188000632a460b7f5a, role_id=ca1a5695f8bd499aa0b8cedaeecd2cc6)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:16.846535 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fb5c75-30d3-4391-acb6-88571c1f22a7 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:16.857494 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 120/239] 10.222.0.15 () {58 vars in 1334 bytes} [Mon Oct 31 12:20:16 2022] PUT /identity/v3/system/users/77cb971b62a249188000632a460b7f5a/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:20:18.538264 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ccbaef-8c5a-4f70-9baa-8a5a3e1ef240 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:18.538758 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ccbaef-8c5a-4f70-9baa-8a5a3e1ef240 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:18.538830 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ccbaef-8c5a-4f70-9baa-8a5a3e1ef240 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:18.540351 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 120/240] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:20:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:20:18.545545 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5529f0d-7b86-42be-b14c-17fceeab0295 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:18.545930 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5529f0d-7b86-42be-b14c-17fceeab0295 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:18.546089 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5529f0d-7b86-42be-b14c-17fceeab0295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:18.565374 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-c5529f0d-7b86-42be-b14c-17fceeab0295 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:20:18.566555 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-c5529f0d-7b86-42be-b14c-17fceeab0295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:20:18.581501 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:18.638404 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 121/241] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:20:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 31 12:20:18.759856 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c0c63f-8487-48cd-8173-d17804bfb2c2 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:18.760296 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c0c63f-8487-48cd-8173-d17804bfb2c2 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:18.760575 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c0c63f-8487-48cd-8173-d17804bfb2c2 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:18.762078 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 121/242] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:20:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:20:18.766934 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50cbecd-5870-4f05-899d-896f37280ec3 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:18.767249 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50cbecd-5870-4f05-899d-896f37280ec3 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:18.767526 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50cbecd-5870-4f05-899d-896f37280ec3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:18.786323 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-c50cbecd-5870-4f05-899d-896f37280ec3 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:20:18.787523 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-c50cbecd-5870-4f05-899d-896f37280ec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:20:18.853675 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 122/243] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:20:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 31 12:20:18.909188 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c0c63f-8487-48cd-8173-d17804bfb2c2 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:19.019871 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c0c63f-8487-48cd-8173-d17804bfb2c2 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:19.058816 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308b11a0-d44c-44b5-a6ff-420227ccbfde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:19.059481 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308b11a0-d44c-44b5-a6ff-420227ccbfde None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:19.059761 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308b11a0-d44c-44b5-a6ff-420227ccbfde None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:19.060032 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308b11a0-d44c-44b5-a6ff-420227ccbfde None admin] PATH_INFO: `/v3/domains/default` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:19.061157 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308b11a0-d44c-44b5-a6ff-420227ccbfde None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:19.062580 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308b11a0-d44c-44b5-a6ff-420227ccbfde None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:19.064385 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 122/244] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Oct 31 12:20:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:19.069767 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50cbecd-5870-4f05-899d-896f37280ec3 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:19.071455 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4857a79-9b60-4d13-84d5-bad0f2ed342f None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:19.071740 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4857a79-9b60-4d13-84d5-bad0f2ed342f None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:19.072014 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4857a79-9b60-4d13-84d5-bad0f2ed342f None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:19.073446 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 123/245] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:20:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:20:19.078587 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40989180-257a-484c-a318-dee8b07c1313 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:19.078865 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40989180-257a-484c-a318-dee8b07c1313 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:19.079136 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40989180-257a-484c-a318-dee8b07c1313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:19.100070 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-40989180-257a-484c-a318-dee8b07c1313 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:20:19.101244 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-40989180-257a-484c-a318-dee8b07c1313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:20:19.154915 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:19.158925 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:19.172858 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 124/246] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:20:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 31 12:20:19.173681 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40989180-257a-484c-a318-dee8b07c1313 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:19.185253 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50cbecd-5870-4f05-899d-896f37280ec3 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:19.227532 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72ca041-6f94-405b-ab09-d73d10dbb4e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:19.228142 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72ca041-6f94-405b-ab09-d73d10dbb4e7 None admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:19.228418 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72ca041-6f94-405b-ab09-d73d10dbb4e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:19.228683 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72ca041-6f94-405b-ab09-d73d10dbb4e7 None admin] PATH_INFO: `/v3/groups` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:19.229304 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72ca041-6f94-405b-ab09-d73d10dbb4e7 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:19.230752 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72ca041-6f94-405b-ab09-d73d10dbb4e7 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:19.247046 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 123/247] 10.222.0.15 () {60 vars in 1134 bytes} [Mon Oct 31 12:20:19 2022] POST /identity/v3/groups => generated 226 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:20:19.290434 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40989180-257a-484c-a318-dee8b07c1313 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:19.330812 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be69c18d-8946-4983-bbe8-4540ff6d4e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:19.331647 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be69c18d-8946-4983-bbe8-4540ff6d4e02 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:19.331931 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be69c18d-8946-4983-bbe8-4540ff6d4e02 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:19.332243 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be69c18d-8946-4983-bbe8-4540ff6d4e02 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:19.344924 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be69c18d-8946-4983-bbe8-4540ff6d4e02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 31 12:20:19.350334 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-be69c18d-8946-4983-bbe8-4540ff6d4e02 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 31 12:20:19.350334 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:19.350334 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:19.350334 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:19.350334 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:19.350334 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:19.350334 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:19.350334 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:19.350334 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:19.350334 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:19.350334 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:19.350334 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:19.350334 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 31 12:20:19.350334 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 31 12:20:19.350334 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 31 12:20:19.350334 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 31 12:20:19.351257 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 31 12:20:19.351257 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 31 12:20:19.351257 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 31 12:20:19.351257 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 31 12:20:19.351257 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:19.351257 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:19.351257 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:19.351257 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:19.351257 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:19.351257 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:19.351257 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:19.351257 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:19.351257 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:19.351257 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:19.351257 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:19.351257 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:19.352029 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:19.352029 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:19.352029 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:19.352029 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:19.352029 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 31 12:20:19.352029 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 31 12:20:19.352029 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 31 12:20:19.352029 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 31 12:20:19.352029 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 31 12:20:19.352029 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 31 12:20:19.352029 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 31 12:20:19.352029 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:20:19.352029 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 125/248] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Oct 31 12:20:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:20:19.358965 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72ca041-6f94-405b-ab09-d73d10dbb4e7 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:19.445186 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be69c18d-8946-4983-bbe8-4540ff6d4e02 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:19.476813 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72ca041-6f94-405b-ab09-d73d10dbb4e7 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:19.509510 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:19.519009 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0044011f-928c-4e41-bac5-9954bf5075f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:19.519659 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0044011f-928c-4e41-bac5-9954bf5075f6 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:19.520390 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0044011f-928c-4e41-bac5-9954bf5075f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:19.520390 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0044011f-928c-4e41-bac5-9954bf5075f6 None admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:19.520950 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0044011f-928c-4e41-bac5-9954bf5075f6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:19.521253 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0044011f-928c-4e41-bac5-9954bf5075f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:19.522619 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0044011f-928c-4e41-bac5-9954bf5075f6 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:19.529110 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:19.534282 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 124/249] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Oct 31 12:20:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:19.539685 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0044011f-928c-4e41-bac5-9954bf5075f6 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:19.575318 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be69c18d-8946-4983-bbe8-4540ff6d4e02 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:19.627873 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564168ee-2ff5-43ee-8628-6779ab984467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:19.628928 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564168ee-2ff5-43ee-8628-6779ab984467 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:19.629409 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564168ee-2ff5-43ee-8628-6779ab984467 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:19.629868 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564168ee-2ff5-43ee-8628-6779ab984467 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:19.644448 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564168ee-2ff5-43ee-8628-6779ab984467 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 31 12:20:19.649439 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-564168ee-2ff5-43ee-8628-6779ab984467 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 31 12:20:19.649439 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:19.649439 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:19.649439 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:19.649439 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:19.649439 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:19.649439 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:19.649439 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:19.649439 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:19.649439 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:19.649439 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:19.649439 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:19.649439 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 31 12:20:19.649439 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 31 12:20:19.649439 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 31 12:20:19.649439 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 31 12:20:19.650480 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 31 12:20:19.650480 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 31 12:20:19.650480 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 31 12:20:19.650480 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 31 12:20:19.650480 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:19.650480 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:19.650480 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:19.650480 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:19.650480 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:19.650480 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:19.650480 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:19.650480 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:19.650480 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:19.650480 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:19.650480 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:19.650480 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:19.651660 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:19.651660 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:19.651660 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:19.651660 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:19.651660 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 31 12:20:19.651660 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 31 12:20:19.651660 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 31 12:20:19.651660 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 31 12:20:19.651660 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 31 12:20:19.651660 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 31 12:20:19.651660 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 31 12:20:19.651660 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:20:19.651660 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 126/250] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Oct 31 12:20:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:20:19.659016 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564168ee-2ff5-43ee-8628-6779ab984467 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:19.669056 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0044011f-928c-4e41-bac5-9954bf5075f6 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:19.709760 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f3a0ef-c251-4ac5-9ead-4481f82cb4a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:19.710308 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f3a0ef-c251-4ac5-9ead-4481f82cb4a3 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:19.710583 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f3a0ef-c251-4ac5-9ead-4481f82cb4a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:19.710849 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f3a0ef-c251-4ac5-9ead-4481f82cb4a3 None admin] PATH_INFO: `/v3/users/eb235f50b9714414a5ba4e523068829b` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:19.712035 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f3a0ef-c251-4ac5-9ead-4481f82cb4a3 None admin] RBAC: Authorizing `identity:get_user(user_id=eb235f50b9714414a5ba4e523068829b)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:19.714350 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f3a0ef-c251-4ac5-9ead-4481f82cb4a3 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:19.716228 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 125/251] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Oct 31 12:20:19 2022] GET /identity/v3/users/eb235f50b9714414a5ba4e523068829b => generated 293 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 31 12:20:19.721792 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f3a0ef-c251-4ac5-9ead-4481f82cb4a3 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:19.801597 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564168ee-2ff5-43ee-8628-6779ab984467 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:19.804230 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:19.811803 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:19.848083 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e86ecd-4b78-4248-ab79-97839ece68a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:19.849039 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e86ecd-4b78-4248-ab79-97839ece68a8 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:19.849533 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e86ecd-4b78-4248-ab79-97839ece68a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:19.850007 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e86ecd-4b78-4248-ab79-97839ece68a8 None admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:19.850773 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e86ecd-4b78-4248-ab79-97839ece68a8 None admin] RBAC: Adding query filter params (name=member) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:19.851201 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e86ecd-4b78-4248-ab79-97839ece68a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:19.852835 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e86ecd-4b78-4248-ab79-97839ece68a8 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:19.860125 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f3a0ef-c251-4ac5-9ead-4481f82cb4a3 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:19.863914 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 127/252] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Oct 31 12:20:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:19.870278 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e86ecd-4b78-4248-ab79-97839ece68a8 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:19.909784 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f93b8f9-11a5-468b-94ca-b94b88e1f1de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:19.910733 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f93b8f9-11a5-468b-94ca-b94b88e1f1de None admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:19.911211 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f93b8f9-11a5-468b-94ca-b94b88e1f1de None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:19.911658 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f93b8f9-11a5-468b-94ca-b94b88e1f1de None admin] PATH_INFO: `/v3/system/users/eb235f50b9714414a5ba4e523068829b/roles/056e0848b1da43f1a1eaae44d5ecddc8` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:19.913335 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f93b8f9-11a5-468b-94ca-b94b88e1f1de None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=eb235f50b9714414a5ba4e523068829b, role_id=056e0848b1da43f1a1eaae44d5ecddc8)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:19.915015 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f93b8f9-11a5-468b-94ca-b94b88e1f1de None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:19.925085 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 126/253] 10.222.0.15 () {58 vars in 1334 bytes} [Mon Oct 31 12:20:19 2022] PUT /identity/v3/system/users/eb235f50b9714414a5ba4e523068829b/roles/056e0848b1da43f1a1eaae44d5ecddc8 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:20:19.988840 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e86ecd-4b78-4248-ab79-97839ece68a8 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:20.043289 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e06da8b-45ec-4002-b7f7-497264a09b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:20.043953 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e06da8b-45ec-4002-b7f7-497264a09b64 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:20.044232 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e06da8b-45ec-4002-b7f7-497264a09b64 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:20.044495 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e06da8b-45ec-4002-b7f7-497264a09b64 None admin] PATH_INFO: `/v3/users/77cb971b62a249188000632a460b7f5a` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:20.046773 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e06da8b-45ec-4002-b7f7-497264a09b64 None admin] RBAC: Authorizing `identity:get_user(user_id=77cb971b62a249188000632a460b7f5a)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:20.048391 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e06da8b-45ec-4002-b7f7-497264a09b64 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:20.051263 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 128/254] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Oct 31 12:20:19 2022] GET /identity/v3/users/77cb971b62a249188000632a460b7f5a => generated 293 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 31 12:20:20.056586 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f93b8f9-11a5-468b-94ca-b94b88e1f1de None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:20.203080 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f93b8f9-11a5-468b-94ca-b94b88e1f1de None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:20.247391 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:20.256807 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce950fe6-82c0-4f90-9085-298316271920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:20.257437 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce950fe6-82c0-4f90-9085-298316271920 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:20.257715 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce950fe6-82c0-4f90-9085-298316271920 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:20.257985 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce950fe6-82c0-4f90-9085-298316271920 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:20.258721 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce950fe6-82c0-4f90-9085-298316271920 None admin] RBAC: Adding query filter params (role.id=ca1a5695f8bd499aa0b8cedaeecd2cc6, user.id=77cb971b62a249188000632a460b7f5a, scope.system=all) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:20.259484 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce950fe6-82c0-4f90-9085-298316271920 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:20.261481 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce950fe6-82c0-4f90-9085-298316271920 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:20.281666 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 127/255] 10.222.0.15 () {56 vars in 1308 bytes} [Mon Oct 31 12:20:20 2022] GET /identity/v3/role_assignments?user.id=77cb971b62a249188000632a460b7f5a&scope.system=all&role.id=ca1a5695f8bd499aa0b8cedaeecd2cc6 => generated 510 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:22.018720 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8660a20b-609f-4d84-9167-8a11ef33d99d None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:22.019595 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8660a20b-609f-4d84-9167-8a11ef33d99d None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:22.020041 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8660a20b-609f-4d84-9167-8a11ef33d99d None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:22.021784 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 129/256] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:20:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:20:22.028351 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9ddee6-8555-443b-bf3d-c9b71eb04af1 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:22.029063 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9ddee6-8555-443b-bf3d-c9b71eb04af1 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:22.029480 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9ddee6-8555-443b-bf3d-c9b71eb04af1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:22.031008 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef302f9b-00b1-415d-bc9f-b5941236d791 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:22.031464 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef302f9b-00b1-415d-bc9f-b5941236d791 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:22.031881 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef302f9b-00b1-415d-bc9f-b5941236d791 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:22.033326 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 130/257] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:20:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:20:22.038623 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81713d82-7c4b-438a-8f0b-98a8ebb48a7f None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:22.039714 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81713d82-7c4b-438a-8f0b-98a8ebb48a7f None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:22.041183 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81713d82-7c4b-438a-8f0b-98a8ebb48a7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:22.069978 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-7f9ddee6-8555-443b-bf3d-c9b71eb04af1 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:20:22.071540 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-7f9ddee6-8555-443b-bf3d-c9b71eb04af1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:20:22.079706 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-81713d82-7c4b-438a-8f0b-98a8ebb48a7f None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:20:22.083213 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-81713d82-7c4b-438a-8f0b-98a8ebb48a7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:20:22.159587 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:22.190448 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 128/258] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:20:22 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 31 12:20:22.195024 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 131/259] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:20:22 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 31 12:20:22.504299 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9ddee6-8555-443b-bf3d-c9b71eb04af1 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:22.516600 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9ddee6-8555-443b-bf3d-c9b71eb04af1 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:22.521632 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5697fb-aa08-4407-9b84-3bf6d0976da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:22.522497 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5697fb-aa08-4407-9b84-3bf6d0976da9 admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:22.522975 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5697fb-aa08-4407-9b84-3bf6d0976da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:22.523529 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5697fb-aa08-4407-9b84-3bf6d0976da9 admin admin] PATH_INFO: `/v3/roles/member` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:22.536517 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5697fb-aa08-4407-9b84-3bf6d0976da9 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 31 12:20:22.541854 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-eb5697fb-aa08-4407-9b84-3bf6d0976da9 admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 31 12:20:22.541854 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:22.541854 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:22.541854 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:22.541854 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:22.541854 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:22.541854 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:22.541854 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:22.541854 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:22.541854 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:22.541854 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:22.541854 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:22.541854 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 31 12:20:22.541854 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 31 12:20:22.541854 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 31 12:20:22.541854 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 31 12:20:22.542735 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 31 12:20:22.542735 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 31 12:20:22.542735 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 31 12:20:22.542735 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 31 12:20:22.542735 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:22.542735 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:22.542735 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:22.542735 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:22.542735 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:22.542735 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:22.542735 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:22.542735 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:22.542735 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:22.542735 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:22.542735 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:22.542735 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:22.543590 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:22.543590 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:22.543590 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:22.543590 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:22.543590 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 31 12:20:22.543590 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 31 12:20:22.543590 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 31 12:20:22.543590 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 31 12:20:22.543590 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 31 12:20:22.543590 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 31 12:20:22.543590 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 31 12:20:22.543590 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:20:22.546158 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 129/260] 10.222.0.15 () {56 vars in 1120 bytes} [Mon Oct 31 12:20:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 42 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:20:22.552131 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81713d82-7c4b-438a-8f0b-98a8ebb48a7f None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:22.562357 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81713d82-7c4b-438a-8f0b-98a8ebb48a7f None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:22.569186 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc64c54-3cd7-44a2-9b0a-cd33963bdd23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:22.570043 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc64c54-3cd7-44a2-9b0a-cd33963bdd23 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:22.570519 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc64c54-3cd7-44a2-9b0a-cd33963bdd23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:22.570982 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc64c54-3cd7-44a2-9b0a-cd33963bdd23 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:22.571941 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc64c54-3cd7-44a2-9b0a-cd33963bdd23 admin admin] RBAC: Adding query filter params (name=member) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:22.572446 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc64c54-3cd7-44a2-9b0a-cd33963bdd23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:22.574529 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: WARNING py.warnings [None req-1bc64c54-3cd7-44a2-9b0a-cd33963bdd23 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 31 12:20:22.574529 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: warnings.warn(msg) Oct 31 12:20:22.574529 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: Oct 31 12:20:22.575921 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc64c54-3cd7-44a2-9b0a-cd33963bdd23 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:22.587261 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 132/261] 10.222.0.15 () {56 vars in 1122 bytes} [Mon Oct 31 12:20:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:22.593485 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5697fb-aa08-4407-9b84-3bf6d0976da9 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:22.608686 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5697fb-aa08-4407-9b84-3bf6d0976da9 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:22.613554 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a96072-a101-437e-a151-9a2f9ab0ab8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:22.614430 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a96072-a101-437e-a151-9a2f9ab0ab8a admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:22.614920 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a96072-a101-437e-a151-9a2f9ab0ab8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:22.615485 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a96072-a101-437e-a151-9a2f9ab0ab8a admin admin] PATH_INFO: `/v3/groups/0ebd842d7a254cfcb86cfa52b46e4826` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:22.625512 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a96072-a101-437e-a151-9a2f9ab0ab8a admin admin] RBAC: Authorizing `identity:get_group(group_id=0ebd842d7a254cfcb86cfa52b46e4826)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:22.627592 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: WARNING py.warnings [None req-94a96072-a101-437e-a151-9a2f9ab0ab8a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 31 12:20:22.627592 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: warnings.warn(msg) Oct 31 12:20:22.627592 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: Oct 31 12:20:22.629183 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a96072-a101-437e-a151-9a2f9ab0ab8a admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:22.631440 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 130/262] 10.222.0.15 () {56 vars in 1201 bytes} [Mon Oct 31 12:20:22 2022] GET /identity/v3/groups/0ebd842d7a254cfcb86cfa52b46e4826 => generated 226 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:22.640494 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc64c54-3cd7-44a2-9b0a-cd33963bdd23 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:22.652879 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc64c54-3cd7-44a2-9b0a-cd33963bdd23 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:22.659805 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc069b2-d352-4532-ad07-7e17f21dc4f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:22.660758 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc069b2-d352-4532-ad07-7e17f21dc4f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:22.661296 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc069b2-d352-4532-ad07-7e17f21dc4f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:22.661769 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc069b2-d352-4532-ad07-7e17f21dc4f7 admin admin] PATH_INFO: `/v3/projects/9bc6131ae0634994bd48138d77878576` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:22.663124 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc069b2-d352-4532-ad07-7e17f21dc4f7 admin admin] RBAC: Authorizing `identity:get_project(project_id=9bc6131ae0634994bd48138d77878576)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:22.665746 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc069b2-d352-4532-ad07-7e17f21dc4f7 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:22.668391 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 133/263] 10.222.0.15 () {56 vars in 1207 bytes} [Mon Oct 31 12:20:22 2022] GET /identity/v3/projects/9bc6131ae0634994bd48138d77878576 => generated 298 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:22.674075 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a96072-a101-437e-a151-9a2f9ab0ab8a admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:22.686243 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a96072-a101-437e-a151-9a2f9ab0ab8a admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:22.692228 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd509ec9-5510-4a5d-ab8a-9a3971d206ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:22.693276 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd509ec9-5510-4a5d-ab8a-9a3971d206ac admin admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:22.693756 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd509ec9-5510-4a5d-ab8a-9a3971d206ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:22.694259 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd509ec9-5510-4a5d-ab8a-9a3971d206ac admin admin] PATH_INFO: `/v3/projects/9bc6131ae0634994bd48138d77878576/groups/0ebd842d7a254cfcb86cfa52b46e4826/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:22.696921 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd509ec9-5510-4a5d-ab8a-9a3971d206ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bc6131ae0634994bd48138d77878576, group_id=0ebd842d7a254cfcb86cfa52b46e4826, role_id=ca1a5695f8bd499aa0b8cedaeecd2cc6)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:22.699377 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: WARNING py.warnings [None req-dd509ec9-5510-4a5d-ab8a-9a3971d206ac admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 31 12:20:22.699377 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: warnings.warn(msg) Oct 31 12:20:22.699377 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: Oct 31 12:20:22.701807 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd509ec9-5510-4a5d-ab8a-9a3971d206ac admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:22.725162 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 131/264] 10.222.0.15 () {58 vars in 1463 bytes} [Mon Oct 31 12:20:22 2022] PUT /identity/v3/projects/9bc6131ae0634994bd48138d77878576/groups/0ebd842d7a254cfcb86cfa52b46e4826/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:20:22.735471 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc069b2-d352-4532-ad07-7e17f21dc4f7 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:22.749143 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc069b2-d352-4532-ad07-7e17f21dc4f7 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:22.822019 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4235d67c-73bb-43d3-8559-2c099d48b699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:22.824191 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4235d67c-73bb-43d3-8559-2c099d48b699 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:22.824679 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4235d67c-73bb-43d3-8559-2c099d48b699 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:22.825179 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4235d67c-73bb-43d3-8559-2c099d48b699 admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:22.842725 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4235d67c-73bb-43d3-8559-2c099d48b699 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 31 12:20:22.848970 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-4235d67c-73bb-43d3-8559-2c099d48b699 admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 31 12:20:22.848970 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:22.848970 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:22.848970 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:22.848970 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:22.848970 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:22.848970 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:22.848970 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:22.848970 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:22.848970 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:22.848970 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:22.848970 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:22.848970 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 31 12:20:22.848970 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 31 12:20:22.848970 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 31 12:20:22.848970 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 31 12:20:22.849926 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 31 12:20:22.849926 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 31 12:20:22.849926 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 31 12:20:22.849926 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 31 12:20:22.849926 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:22.849926 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:22.849926 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:22.849926 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:22.849926 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:22.849926 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:22.849926 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:22.849926 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:22.849926 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:22.849926 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:22.849926 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:22.849926 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:22.850716 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:22.850716 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:22.850716 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:22.850716 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:22.850716 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 31 12:20:22.850716 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 31 12:20:22.850716 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 31 12:20:22.850716 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 31 12:20:22.850716 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 31 12:20:22.850716 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 31 12:20:22.850716 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 31 12:20:22.850716 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:20:22.852874 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 134/265] 10.222.0.15 () {56 vars in 1135 bytes} [Mon Oct 31 12:20:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 118 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:20:22.859250 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd509ec9-5510-4a5d-ab8a-9a3971d206ac admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:22.871478 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd509ec9-5510-4a5d-ab8a-9a3971d206ac admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:22.877966 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4a484d-b8bc-445b-be2b-1bdf98938db0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:22.878824 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4a484d-b8bc-445b-be2b-1bdf98938db0 admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:22.879304 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4a484d-b8bc-445b-be2b-1bdf98938db0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:22.879717 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4a484d-b8bc-445b-be2b-1bdf98938db0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:22.880612 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4a484d-b8bc-445b-be2b-1bdf98938db0 admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:22.881088 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4a484d-b8bc-445b-be2b-1bdf98938db0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:22.882251 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: WARNING py.warnings [None req-ca4a484d-b8bc-445b-be2b-1bdf98938db0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 31 12:20:22.882251 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: warnings.warn(msg) Oct 31 12:20:22.882251 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: Oct 31 12:20:22.883525 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4a484d-b8bc-445b-be2b-1bdf98938db0 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:22.895194 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 132/266] 10.222.0.15 () {56 vars in 1132 bytes} [Mon Oct 31 12:20:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:22.901858 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4235d67c-73bb-43d3-8559-2c099d48b699 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:22.916853 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4235d67c-73bb-43d3-8559-2c099d48b699 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:22.929252 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d01511a-781f-427d-ba3f-af49c4ef00de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:22.930161 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d01511a-781f-427d-ba3f-af49c4ef00de admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:22.930758 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d01511a-781f-427d-ba3f-af49c4ef00de admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:22.931226 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d01511a-781f-427d-ba3f-af49c4ef00de admin admin] PATH_INFO: `/v3/groups/0ebd842d7a254cfcb86cfa52b46e4826` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:22.933238 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d01511a-781f-427d-ba3f-af49c4ef00de admin admin] RBAC: Authorizing `identity:get_group(group_id=0ebd842d7a254cfcb86cfa52b46e4826)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:22.934485 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: WARNING py.warnings [None req-2d01511a-781f-427d-ba3f-af49c4ef00de admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 31 12:20:22.934485 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: warnings.warn(msg) Oct 31 12:20:22.934485 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: Oct 31 12:20:22.936152 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d01511a-781f-427d-ba3f-af49c4ef00de admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:22.938024 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 135/267] 10.222.0.15 () {56 vars in 1201 bytes} [Mon Oct 31 12:20:22 2022] GET /identity/v3/groups/0ebd842d7a254cfcb86cfa52b46e4826 => generated 226 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:22.943774 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4a484d-b8bc-445b-be2b-1bdf98938db0 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:22.957447 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4a484d-b8bc-445b-be2b-1bdf98938db0 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:22.962767 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1c7238-9061-49c9-8889-91029c1f7eb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:22.964647 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1c7238-9061-49c9-8889-91029c1f7eb6 admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:22.965086 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1c7238-9061-49c9-8889-91029c1f7eb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:22.965641 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1c7238-9061-49c9-8889-91029c1f7eb6 admin admin] PATH_INFO: `/v3/projects/9bc6131ae0634994bd48138d77878576` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:22.968043 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1c7238-9061-49c9-8889-91029c1f7eb6 admin admin] RBAC: Authorizing `identity:get_project(project_id=9bc6131ae0634994bd48138d77878576)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:22.970516 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1c7238-9061-49c9-8889-91029c1f7eb6 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:22.972533 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 133/268] 10.222.0.15 () {56 vars in 1207 bytes} [Mon Oct 31 12:20:22 2022] GET /identity/v3/projects/9bc6131ae0634994bd48138d77878576 => generated 298 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:22.978399 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d01511a-781f-427d-ba3f-af49c4ef00de admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:22.992063 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d01511a-781f-427d-ba3f-af49c4ef00de admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:22.998568 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a6adbd-2952-4a1a-bc2f-c294a9843553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:22.999600 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a6adbd-2952-4a1a-bc2f-c294a9843553 admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:23.000072 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a6adbd-2952-4a1a-bc2f-c294a9843553 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:23.000491 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a6adbd-2952-4a1a-bc2f-c294a9843553 admin admin] PATH_INFO: `/v3/projects/9bc6131ae0634994bd48138d77878576/groups/0ebd842d7a254cfcb86cfa52b46e4826/roles/65728d8bb0e546788426da322c34baba` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:23.003062 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a6adbd-2952-4a1a-bc2f-c294a9843553 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bc6131ae0634994bd48138d77878576, group_id=0ebd842d7a254cfcb86cfa52b46e4826, role_id=65728d8bb0e546788426da322c34baba)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:23.004370 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: WARNING py.warnings [None req-e3a6adbd-2952-4a1a-bc2f-c294a9843553 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 31 12:20:23.004370 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: warnings.warn(msg) Oct 31 12:20:23.004370 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: Oct 31 12:20:23.006823 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a6adbd-2952-4a1a-bc2f-c294a9843553 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:23.045237 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 136/269] 10.222.0.15 () {58 vars in 1463 bytes} [Mon Oct 31 12:20:22 2022] PUT /identity/v3/projects/9bc6131ae0634994bd48138d77878576/groups/0ebd842d7a254cfcb86cfa52b46e4826/roles/65728d8bb0e546788426da322c34baba => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:20:23.099504 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946ec942-208d-40e1-8157-64809bb797f9 None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:23.100393 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946ec942-208d-40e1-8157-64809bb797f9 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:23.100817 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946ec942-208d-40e1-8157-64809bb797f9 None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:23.102780 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 134/270] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:20:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:20:23.108536 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff42b20e-4d81-4046-b6eb-1059f6bdf638 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:23.109042 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff42b20e-4d81-4046-b6eb-1059f6bdf638 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:23.109485 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff42b20e-4d81-4046-b6eb-1059f6bdf638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:23.132370 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-ff42b20e-4d81-4046-b6eb-1059f6bdf638 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:20:23.133865 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-ff42b20e-4d81-4046-b6eb-1059f6bdf638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:20:23.208841 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 137/271] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:20:23 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 31 12:20:23.424659 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defbae66-92e5-414b-afa4-95dc0e970755 None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:23.425173 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defbae66-92e5-414b-afa4-95dc0e970755 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:23.425592 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defbae66-92e5-414b-afa4-95dc0e970755 None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:23.427095 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 135/272] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:20:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:20:23.433838 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fe2783-8000-4038-a9eb-0d7ea36c50ac None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:23.434308 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fe2783-8000-4038-a9eb-0d7ea36c50ac None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:23.434810 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fe2783-8000-4038-a9eb-0d7ea36c50ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:23.460243 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-66fe2783-8000-4038-a9eb-0d7ea36c50ac None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:20:23.461742 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-66fe2783-8000-4038-a9eb-0d7ea36c50ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:20:23.501259 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:23.547200 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 138/273] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:20:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 31 12:20:23.573489 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defbae66-92e5-414b-afa4-95dc0e970755 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:23.589389 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defbae66-92e5-414b-afa4-95dc0e970755 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:23.597121 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260e3229-1956-4b04-9eb0-92361d0b4b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:23.597766 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260e3229-1956-4b04-9eb0-92361d0b4b50 admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:23.598046 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260e3229-1956-4b04-9eb0-92361d0b4b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:23.598311 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260e3229-1956-4b04-9eb0-92361d0b4b50 admin admin] PATH_INFO: `/v3/roles/member` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:23.615685 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260e3229-1956-4b04-9eb0-92361d0b4b50 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 31 12:20:23.621345 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-260e3229-1956-4b04-9eb0-92361d0b4b50 admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 31 12:20:23.621345 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:23.621345 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:23.621345 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:23.621345 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:23.621345 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:23.621345 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:23.621345 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:23.621345 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:23.621345 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:23.621345 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:23.621345 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:23.621345 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 31 12:20:23.621345 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 31 12:20:23.621345 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 31 12:20:23.621345 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 31 12:20:23.622244 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 31 12:20:23.622244 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 31 12:20:23.622244 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 31 12:20:23.622244 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 31 12:20:23.622244 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:23.622244 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:23.622244 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:23.622244 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:23.622244 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:23.622244 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:23.622244 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:23.622244 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:23.622244 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:23.622244 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:23.622244 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:23.622244 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:23.623047 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:23.623047 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:23.623047 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:23.623047 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:23.623047 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 31 12:20:23.623047 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 31 12:20:23.623047 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 31 12:20:23.623047 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 31 12:20:23.623047 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 31 12:20:23.623047 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 31 12:20:23.623047 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 31 12:20:23.623047 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:20:23.623047 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 136/274] 10.222.0.15 () {56 vars in 1120 bytes} [Mon Oct 31 12:20:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 50 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:20:23.628622 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fe2783-8000-4038-a9eb-0d7ea36c50ac None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:23.645177 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fe2783-8000-4038-a9eb-0d7ea36c50ac None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:23.655001 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37856e1f-5b8c-4772-93ff-cc3d3eaf8404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:23.655650 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37856e1f-5b8c-4772-93ff-cc3d3eaf8404 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:23.655929 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37856e1f-5b8c-4772-93ff-cc3d3eaf8404 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:23.656200 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37856e1f-5b8c-4772-93ff-cc3d3eaf8404 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:23.656870 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37856e1f-5b8c-4772-93ff-cc3d3eaf8404 admin admin] RBAC: Adding query filter params (name=member) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:23.657171 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37856e1f-5b8c-4772-93ff-cc3d3eaf8404 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:23.658776 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37856e1f-5b8c-4772-93ff-cc3d3eaf8404 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:23.672297 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 139/275] 10.222.0.15 () {56 vars in 1122 bytes} [Mon Oct 31 12:20:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:23.677917 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260e3229-1956-4b04-9eb0-92361d0b4b50 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:23.692394 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260e3229-1956-4b04-9eb0-92361d0b4b50 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:23.702840 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccdc249-468b-4d70-8150-21c46c3d26fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:23.703505 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccdc249-468b-4d70-8150-21c46c3d26fa admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:23.703786 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccdc249-468b-4d70-8150-21c46c3d26fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:23.704070 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccdc249-468b-4d70-8150-21c46c3d26fa admin admin] PATH_INFO: `/v3/groups/0ebd842d7a254cfcb86cfa52b46e4826` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:23.707817 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccdc249-468b-4d70-8150-21c46c3d26fa admin admin] RBAC: Authorizing `identity:get_group(group_id=0ebd842d7a254cfcb86cfa52b46e4826)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:23.709750 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccdc249-468b-4d70-8150-21c46c3d26fa admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:23.711682 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 137/276] 10.222.0.15 () {56 vars in 1201 bytes} [Mon Oct 31 12:20:23 2022] GET /identity/v3/groups/0ebd842d7a254cfcb86cfa52b46e4826 => generated 226 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:23.717010 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37856e1f-5b8c-4772-93ff-cc3d3eaf8404 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:23.732474 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37856e1f-5b8c-4772-93ff-cc3d3eaf8404 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:23.742841 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f44a7b0-ca3e-41d2-bc0b-af27733152ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:23.743509 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f44a7b0-ca3e-41d2-bc0b-af27733152ba admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:23.743788 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f44a7b0-ca3e-41d2-bc0b-af27733152ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:23.744055 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f44a7b0-ca3e-41d2-bc0b-af27733152ba admin admin] PATH_INFO: `/v3/projects/574ce97a765a42729f79bf108a26330a` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:23.751898 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f44a7b0-ca3e-41d2-bc0b-af27733152ba admin admin] RBAC: Authorizing `identity:get_project(project_id=574ce97a765a42729f79bf108a26330a)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:23.754161 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f44a7b0-ca3e-41d2-bc0b-af27733152ba admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:23.756153 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 140/277] 10.222.0.15 () {56 vars in 1207 bytes} [Mon Oct 31 12:20:23 2022] GET /identity/v3/projects/574ce97a765a42729f79bf108a26330a => generated 302 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:23.761994 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccdc249-468b-4d70-8150-21c46c3d26fa admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:23.776768 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f91cb1-ef5c-49f2-991d-89655ca42076 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:23.777237 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f91cb1-ef5c-49f2-991d-89655ca42076 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:23.777519 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f91cb1-ef5c-49f2-991d-89655ca42076 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:23.777688 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccdc249-468b-4d70-8150-21c46c3d26fa admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:23.779128 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 141/278] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:20:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 31 12:20:23.784339 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa43c23-1b37-4c7e-b7a1-148af669ee53 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:23.784619 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa43c23-1b37-4c7e-b7a1-148af669ee53 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:23.784982 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa43c23-1b37-4c7e-b7a1-148af669ee53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:23.785511 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d0c744-29d4-44ce-b888-3627003a9cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:23.786154 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d0c744-29d4-44ce-b888-3627003a9cb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:23.786436 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d0c744-29d4-44ce-b888-3627003a9cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:23.786708 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d0c744-29d4-44ce-b888-3627003a9cb7 admin admin] PATH_INFO: `/v3/projects/574ce97a765a42729f79bf108a26330a/groups/0ebd842d7a254cfcb86cfa52b46e4826/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:23.792248 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d0c744-29d4-44ce-b888-3627003a9cb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=574ce97a765a42729f79bf108a26330a, group_id=0ebd842d7a254cfcb86cfa52b46e4826, role_id=ca1a5695f8bd499aa0b8cedaeecd2cc6)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:23.795251 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d0c744-29d4-44ce-b888-3627003a9cb7 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:23.812872 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-8aa43c23-1b37-4c7e-b7a1-148af669ee53 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:20:23.814057 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-8aa43c23-1b37-4c7e-b7a1-148af669ee53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:20:23.825313 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 138/279] 10.222.0.15 () {58 vars in 1463 bytes} [Mon Oct 31 12:20:23 2022] PUT /identity/v3/projects/574ce97a765a42729f79bf108a26330a/groups/0ebd842d7a254cfcb86cfa52b46e4826/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:20:23.893469 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 142/280] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:20:23 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 31 12:20:23.944281 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d0c744-29d4-44ce-b888-3627003a9cb7 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:24.036538 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:24.099661 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d0c744-29d4-44ce-b888-3627003a9cb7 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:24.163033 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215d90e8-7ca8-4f47-93e6-cccd40d395aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:24.163706 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215d90e8-7ca8-4f47-93e6-cccd40d395aa None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:24.163986 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215d90e8-7ca8-4f47-93e6-cccd40d395aa None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:24.164259 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215d90e8-7ca8-4f47-93e6-cccd40d395aa None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:24.177139 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215d90e8-7ca8-4f47-93e6-cccd40d395aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 31 12:20:24.182790 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-215d90e8-7ca8-4f47-93e6-cccd40d395aa None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 31 12:20:24.182790 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:24.182790 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:24.182790 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:24.182790 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:24.182790 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:24.182790 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:24.182790 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:24.182790 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:24.182790 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:24.182790 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:24.182790 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:24.182790 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 31 12:20:24.182790 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 31 12:20:24.182790 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 31 12:20:24.182790 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 31 12:20:24.183699 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 31 12:20:24.183699 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 31 12:20:24.183699 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 31 12:20:24.183699 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 31 12:20:24.183699 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:24.183699 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:24.183699 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:24.183699 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:24.183699 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:24.183699 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:24.183699 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:24.183699 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:24.183699 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:24.183699 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:24.183699 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:24.183699 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:24.184494 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:24.184494 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:24.184494 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:24.184494 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:24.184494 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 31 12:20:24.184494 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 31 12:20:24.184494 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 31 12:20:24.184494 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 31 12:20:24.184494 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 31 12:20:24.184494 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 31 12:20:24.184494 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 31 12:20:24.184494 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:20:24.184494 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 139/281] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Oct 31 12:20:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:20:24.189718 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa43c23-1b37-4c7e-b7a1-148af669ee53 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:24.302946 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:24.311907 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa43c23-1b37-4c7e-b7a1-148af669ee53 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:24.350778 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6cdfab-69cb-4b9e-98d6-dbfa07a42cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:24.351423 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6cdfab-69cb-4b9e-98d6-dbfa07a42cbe None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:24.351736 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6cdfab-69cb-4b9e-98d6-dbfa07a42cbe None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:24.352005 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6cdfab-69cb-4b9e-98d6-dbfa07a42cbe None admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:24.352671 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6cdfab-69cb-4b9e-98d6-dbfa07a42cbe None admin] RBAC: Adding query filter params (name=reader) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:24.352969 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6cdfab-69cb-4b9e-98d6-dbfa07a42cbe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:24.355522 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215d90e8-7ca8-4f47-93e6-cccd40d395aa None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:24.361422 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6cdfab-69cb-4b9e-98d6-dbfa07a42cbe None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:24.370240 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 143/282] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Oct 31 12:20:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:24.373627 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215d90e8-7ca8-4f47-93e6-cccd40d395aa None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:24.376042 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6cdfab-69cb-4b9e-98d6-dbfa07a42cbe None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:24.381251 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f03872-1a71-4eaf-b144-778ada6cf192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:24.381989 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f03872-1a71-4eaf-b144-778ada6cf192 admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:24.382193 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f03872-1a71-4eaf-b144-778ada6cf192 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:24.382437 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f03872-1a71-4eaf-b144-778ada6cf192 admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:24.402555 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f03872-1a71-4eaf-b144-778ada6cf192 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 31 12:20:24.407886 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-56f03872-1a71-4eaf-b144-778ada6cf192 admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 31 12:20:24.407886 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:24.407886 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:24.407886 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:24.407886 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:24.407886 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:24.407886 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:24.407886 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:24.407886 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:24.407886 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:24.407886 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:24.407886 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:24.407886 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 31 12:20:24.407886 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 31 12:20:24.407886 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 31 12:20:24.407886 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 31 12:20:24.408777 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 31 12:20:24.408777 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 31 12:20:24.408777 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 31 12:20:24.408777 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 31 12:20:24.408777 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:24.408777 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:24.408777 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:24.408777 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:24.408777 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:24.408777 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:24.408777 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:24.408777 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:24.408777 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:24.408777 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:24.408777 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:24.408777 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:24.409567 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:24.409567 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:24.409567 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:24.409567 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:24.409567 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 31 12:20:24.409567 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 31 12:20:24.409567 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 31 12:20:24.409567 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 31 12:20:24.409567 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 31 12:20:24.409567 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 31 12:20:24.409567 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 31 12:20:24.409567 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:20:24.410172 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 140/283] 10.222.0.15 () {56 vars in 1135 bytes} [Mon Oct 31 12:20:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 55 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:20:24.420002 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f03872-1a71-4eaf-b144-778ada6cf192 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:24.437351 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f03872-1a71-4eaf-b144-778ada6cf192 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:24.449102 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d95e2d-43e0-47e8-9c99-ef156b68f73c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:24.450773 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d95e2d-43e0-47e8-9c99-ef156b68f73c admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:24.451349 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d95e2d-43e0-47e8-9c99-ef156b68f73c admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:24.452359 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d95e2d-43e0-47e8-9c99-ef156b68f73c admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:24.453792 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d95e2d-43e0-47e8-9c99-ef156b68f73c admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:24.455011 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d95e2d-43e0-47e8-9c99-ef156b68f73c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:24.459446 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d95e2d-43e0-47e8-9c99-ef156b68f73c admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:24.474402 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 141/284] 10.222.0.15 () {56 vars in 1132 bytes} [Mon Oct 31 12:20:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:24.481465 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d95e2d-43e0-47e8-9c99-ef156b68f73c admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:24.498193 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d95e2d-43e0-47e8-9c99-ef156b68f73c admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:24.508304 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca85cd16-b0a8-4af0-83fe-338b80adb01e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:24.509906 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca85cd16-b0a8-4af0-83fe-338b80adb01e admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:24.510601 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca85cd16-b0a8-4af0-83fe-338b80adb01e admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:24.511210 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca85cd16-b0a8-4af0-83fe-338b80adb01e admin admin] PATH_INFO: `/v3/groups/0ebd842d7a254cfcb86cfa52b46e4826` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:24.513225 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca85cd16-b0a8-4af0-83fe-338b80adb01e admin admin] RBAC: Authorizing `identity:get_group(group_id=0ebd842d7a254cfcb86cfa52b46e4826)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:24.518362 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca85cd16-b0a8-4af0-83fe-338b80adb01e admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:24.522219 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 142/285] 10.222.0.15 () {56 vars in 1201 bytes} [Mon Oct 31 12:20:24 2022] GET /identity/v3/groups/0ebd842d7a254cfcb86cfa52b46e4826 => generated 226 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:24.529634 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca85cd16-b0a8-4af0-83fe-338b80adb01e admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:24.542842 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca85cd16-b0a8-4af0-83fe-338b80adb01e admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:24.554583 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3bc6f7-302a-4ccb-8a08-f5ac79a29db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:24.557512 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3bc6f7-302a-4ccb-8a08-f5ac79a29db2 admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:24.558967 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3bc6f7-302a-4ccb-8a08-f5ac79a29db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:24.559463 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3bc6f7-302a-4ccb-8a08-f5ac79a29db2 admin admin] PATH_INFO: `/v3/projects/574ce97a765a42729f79bf108a26330a` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:24.562633 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3bc6f7-302a-4ccb-8a08-f5ac79a29db2 admin admin] RBAC: Authorizing `identity:get_project(project_id=574ce97a765a42729f79bf108a26330a)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:24.566211 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3bc6f7-302a-4ccb-8a08-f5ac79a29db2 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:24.569693 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 143/286] 10.222.0.15 () {56 vars in 1207 bytes} [Mon Oct 31 12:20:24 2022] GET /identity/v3/projects/574ce97a765a42729f79bf108a26330a => generated 302 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 31 12:20:24.575469 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3bc6f7-302a-4ccb-8a08-f5ac79a29db2 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:24.591285 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6cdfab-69cb-4b9e-98d6-dbfa07a42cbe None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:24.597555 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3bc6f7-302a-4ccb-8a08-f5ac79a29db2 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:24.623939 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4960d92-3dcd-4b13-a971-ce1b8f6fbefa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:24.624630 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4960d92-3dcd-4b13-a971-ce1b8f6fbefa admin admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:24.624919 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4960d92-3dcd-4b13-a971-ce1b8f6fbefa admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:24.625203 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4960d92-3dcd-4b13-a971-ce1b8f6fbefa admin admin] PATH_INFO: `/v3/projects/574ce97a765a42729f79bf108a26330a/groups/0ebd842d7a254cfcb86cfa52b46e4826/roles/65728d8bb0e546788426da322c34baba` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:24.633030 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4960d92-3dcd-4b13-a971-ce1b8f6fbefa admin admin] RBAC: Authorizing `identity:create_grant(project_id=574ce97a765a42729f79bf108a26330a, group_id=0ebd842d7a254cfcb86cfa52b46e4826, role_id=65728d8bb0e546788426da322c34baba)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:24.635893 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4960d92-3dcd-4b13-a971-ce1b8f6fbefa admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:24.650035 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695c3c2c-06f6-400b-b6ea-95d4c1d14ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:24.651819 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695c3c2c-06f6-400b-b6ea-95d4c1d14ee1 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:24.652295 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695c3c2c-06f6-400b-b6ea-95d4c1d14ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:24.652810 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695c3c2c-06f6-400b-b6ea-95d4c1d14ee1 None admin] PATH_INFO: `/v3/users/eb235f50b9714414a5ba4e523068829b` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:24.655829 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695c3c2c-06f6-400b-b6ea-95d4c1d14ee1 None admin] RBAC: Authorizing `identity:get_user(user_id=eb235f50b9714414a5ba4e523068829b)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:24.657609 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695c3c2c-06f6-400b-b6ea-95d4c1d14ee1 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:24.661028 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 144/287] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Oct 31 12:20:24 2022] GET /identity/v3/users/eb235f50b9714414a5ba4e523068829b => generated 293 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 31 12:20:24.668131 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695c3c2c-06f6-400b-b6ea-95d4c1d14ee1 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:24.679846 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 144/288] 10.222.0.15 () {58 vars in 1463 bytes} [Mon Oct 31 12:20:24 2022] PUT /identity/v3/projects/574ce97a765a42729f79bf108a26330a/groups/0ebd842d7a254cfcb86cfa52b46e4826/roles/65728d8bb0e546788426da322c34baba => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 31 12:20:24.717445 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:24.814941 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695c3c2c-06f6-400b-b6ea-95d4c1d14ee1 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:24.858537 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfc5678-270b-4318-a40f-4b91fa5c48f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:24.859354 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfc5678-270b-4318-a40f-4b91fa5c48f8 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:24.859730 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfc5678-270b-4318-a40f-4b91fa5c48f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:24.860116 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfc5678-270b-4318-a40f-4b91fa5c48f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:24.860973 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfc5678-270b-4318-a40f-4b91fa5c48f8 None admin] RBAC: Adding query filter params (role.id=056e0848b1da43f1a1eaae44d5ecddc8, user.id=eb235f50b9714414a5ba4e523068829b, scope.system=all) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:24.861380 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfc5678-270b-4318-a40f-4b91fa5c48f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:24.863095 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfc5678-270b-4318-a40f-4b91fa5c48f8 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:24.878420 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 145/289] 10.222.0.15 () {56 vars in 1308 bytes} [Mon Oct 31 12:20:24 2022] GET /identity/v3/role_assignments?user.id=eb235f50b9714414a5ba4e523068829b&scope.system=all&role.id=056e0848b1da43f1a1eaae44d5ecddc8 => generated 510 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:24.984675 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e623f62-355f-4d67-b7b1-66e20392f9e6 None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:24.984962 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e623f62-355f-4d67-b7b1-66e20392f9e6 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:24.985240 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e623f62-355f-4d67-b7b1-66e20392f9e6 None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:24.986814 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 145/290] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:20:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:20:24.995829 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd98646-465b-4b0f-a898-f1ad76b78f30 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:24.996289 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd98646-465b-4b0f-a898-f1ad76b78f30 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:24.996667 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd98646-465b-4b0f-a898-f1ad76b78f30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:25.034361 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-efd98646-465b-4b0f-a898-f1ad76b78f30 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:20:25.035835 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-efd98646-465b-4b0f-a898-f1ad76b78f30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:20:25.104984 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 146/291] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:20:24 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 31 12:20:25.499143 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e623f62-355f-4d67-b7b1-66e20392f9e6 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:25.517296 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e623f62-355f-4d67-b7b1-66e20392f9e6 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:25.521907 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ee5e83-0e4a-4a4e-95a9-5b7b1d4bc397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:25.522529 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ee5e83-0e4a-4a4e-95a9-5b7b1d4bc397 admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:25.522808 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ee5e83-0e4a-4a4e-95a9-5b7b1d4bc397 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:25.523076 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ee5e83-0e4a-4a4e-95a9-5b7b1d4bc397 admin admin] PATH_INFO: `/v3/roles/admin` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:25.540329 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ee5e83-0e4a-4a4e-95a9-5b7b1d4bc397 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 31 12:20:25.545543 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-c5ee5e83-0e4a-4a4e-95a9-5b7b1d4bc397 admin admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 31 12:20:25.545543 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:25.545543 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:25.545543 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:25.545543 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:25.545543 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:25.545543 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:25.545543 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:25.545543 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:25.545543 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:25.545543 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:25.545543 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:25.545543 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 31 12:20:25.545543 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 31 12:20:25.545543 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 31 12:20:25.545543 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 31 12:20:25.546403 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 31 12:20:25.546403 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 31 12:20:25.546403 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 31 12:20:25.546403 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 31 12:20:25.546403 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:25.546403 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:25.546403 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:25.546403 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:25.546403 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:25.546403 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:25.546403 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:25.546403 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:25.546403 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:25.546403 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:25.546403 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:25.546403 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:25.547598 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:25.547598 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:25.547598 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:25.547598 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:25.547598 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 31 12:20:25.547598 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 31 12:20:25.547598 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 31 12:20:25.547598 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 31 12:20:25.547598 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 31 12:20:25.547598 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 31 12:20:25.547598 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 31 12:20:25.547598 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:20:25.547598 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 146/292] 10.222.0.15 () {56 vars in 1117 bytes} [Mon Oct 31 12:20:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 48 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:20:25.552418 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd98646-465b-4b0f-a898-f1ad76b78f30 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:25.568325 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd98646-465b-4b0f-a898-f1ad76b78f30 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:25.574496 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b890aa9-365e-4889-aacb-bebf2981d299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:25.575102 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b890aa9-365e-4889-aacb-bebf2981d299 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:25.575409 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b890aa9-365e-4889-aacb-bebf2981d299 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:25.575681 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b890aa9-365e-4889-aacb-bebf2981d299 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:25.576336 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b890aa9-365e-4889-aacb-bebf2981d299 admin admin] RBAC: Adding query filter params (name=admin) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:25.576635 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b890aa9-365e-4889-aacb-bebf2981d299 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:25.578219 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b890aa9-365e-4889-aacb-bebf2981d299 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:25.587984 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 147/293] 10.222.0.15 () {56 vars in 1120 bytes} [Mon Oct 31 12:20:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:25.594091 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ee5e83-0e4a-4a4e-95a9-5b7b1d4bc397 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:25.606303 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ee5e83-0e4a-4a4e-95a9-5b7b1d4bc397 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:25.610672 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12a162c-271c-4557-af20-a6a979df61fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:25.611393 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12a162c-271c-4557-af20-a6a979df61fb admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:25.611669 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12a162c-271c-4557-af20-a6a979df61fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:25.611976 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12a162c-271c-4557-af20-a6a979df61fb admin admin] PATH_INFO: `/v3/groups/c7526800273345768325c7449c0812c0` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:25.617071 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12a162c-271c-4557-af20-a6a979df61fb admin admin] RBAC: Authorizing `identity:get_group(group_id=c7526800273345768325c7449c0812c0)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:25.618980 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12a162c-271c-4557-af20-a6a979df61fb admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:25.620719 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 147/294] 10.222.0.15 () {56 vars in 1201 bytes} [Mon Oct 31 12:20:25 2022] GET /identity/v3/groups/c7526800273345768325c7449c0812c0 => generated 229 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:25.628251 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b890aa9-365e-4889-aacb-bebf2981d299 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:25.639424 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b890aa9-365e-4889-aacb-bebf2981d299 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:25.644138 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c172fb-b066-46f8-bae2-d57df4f1c6eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:25.644760 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c172fb-b066-46f8-bae2-d57df4f1c6eb admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:25.645085 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c172fb-b066-46f8-bae2-d57df4f1c6eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:25.645351 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c172fb-b066-46f8-bae2-d57df4f1c6eb admin admin] PATH_INFO: `/v3/projects/20cfeec04f5e4394b2ab5ceaa7d9c63a` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:25.646445 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c172fb-b066-46f8-bae2-d57df4f1c6eb admin admin] RBAC: Authorizing `identity:get_project(project_id=20cfeec04f5e4394b2ab5ceaa7d9c63a)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:25.648394 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c172fb-b066-46f8-bae2-d57df4f1c6eb admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:25.650328 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 148/295] 10.222.0.15 () {56 vars in 1207 bytes} [Mon Oct 31 12:20:25 2022] GET /identity/v3/projects/20cfeec04f5e4394b2ab5ceaa7d9c63a => generated 344 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:25.656075 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12a162c-271c-4557-af20-a6a979df61fb admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:25.667098 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12a162c-271c-4557-af20-a6a979df61fb admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:25.674218 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496c0f5c-43e3-4a43-a039-fd1778a29852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:25.674937 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496c0f5c-43e3-4a43-a039-fd1778a29852 admin admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:25.675285 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496c0f5c-43e3-4a43-a039-fd1778a29852 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:25.675510 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496c0f5c-43e3-4a43-a039-fd1778a29852 admin admin] PATH_INFO: `/v3/projects/20cfeec04f5e4394b2ab5ceaa7d9c63a/groups/c7526800273345768325c7449c0812c0/roles/176c07b4fe3b4537bdfeb4be21b379c5` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:25.677681 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496c0f5c-43e3-4a43-a039-fd1778a29852 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20cfeec04f5e4394b2ab5ceaa7d9c63a, group_id=c7526800273345768325c7449c0812c0, role_id=176c07b4fe3b4537bdfeb4be21b379c5)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:25.680629 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496c0f5c-43e3-4a43-a039-fd1778a29852 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:25.703730 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 148/296] 10.222.0.15 () {58 vars in 1463 bytes} [Mon Oct 31 12:20:25 2022] PUT /identity/v3/projects/20cfeec04f5e4394b2ab5ceaa7d9c63a/groups/c7526800273345768325c7449c0812c0/roles/176c07b4fe3b4537bdfeb4be21b379c5 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:20:26.379831 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9b4bda-9e03-4cfe-93e5-bd24896b547c None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:26.380477 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9b4bda-9e03-4cfe-93e5-bd24896b547c None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:26.380954 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9b4bda-9e03-4cfe-93e5-bd24896b547c None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:26.382522 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 149/297] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:20:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:20:26.388983 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8be56f8-d7c4-4b98-b6ed-7168cecab937 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:26.389643 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8be56f8-d7c4-4b98-b6ed-7168cecab937 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:26.390109 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8be56f8-d7c4-4b98-b6ed-7168cecab937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:26.420217 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-b8be56f8-d7c4-4b98-b6ed-7168cecab937 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:20:26.421744 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-b8be56f8-d7c4-4b98-b6ed-7168cecab937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:20:26.476769 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:26.513376 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 149/298] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:20:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 31 12:20:26.709707 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e9600d-28b0-46d0-816b-3baf89b94d2f None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:26.710622 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e9600d-28b0-46d0-816b-3baf89b94d2f None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:26.711107 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e9600d-28b0-46d0-816b-3baf89b94d2f None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:26.712753 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 150/299] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:20:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:20:26.718482 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dc6241-b169-4e9f-a162-5f00f285abd6 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:26.719007 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dc6241-b169-4e9f-a162-5f00f285abd6 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:26.719496 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dc6241-b169-4e9f-a162-5f00f285abd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:26.743797 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-c6dc6241-b169-4e9f-a162-5f00f285abd6 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:20:26.746148 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-c6dc6241-b169-4e9f-a162-5f00f285abd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:20:26.839705 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e9600d-28b0-46d0-816b-3baf89b94d2f None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:26.841689 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 150/300] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:20:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 31 12:20:26.875648 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:26.984964 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e9600d-28b0-46d0-816b-3baf89b94d2f None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:27.024068 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68788d78-76ae-4916-8ba9-65a95d50ed80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:27.024626 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68788d78-76ae-4916-8ba9-65a95d50ed80 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:27.024903 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68788d78-76ae-4916-8ba9-65a95d50ed80 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:27.025256 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68788d78-76ae-4916-8ba9-65a95d50ed80 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:27.039248 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68788d78-76ae-4916-8ba9-65a95d50ed80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 31 12:20:27.041859 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-68788d78-76ae-4916-8ba9-65a95d50ed80 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 31 12:20:27.041859 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:27.041859 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:27.041859 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:27.041859 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:27.041859 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:27.041859 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:27.041859 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:27.041859 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:27.041859 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:27.041859 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:27.041859 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:27.041859 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 31 12:20:27.041859 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 31 12:20:27.041859 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 31 12:20:27.041859 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 31 12:20:27.042722 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 31 12:20:27.042722 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 31 12:20:27.042722 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 31 12:20:27.042722 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 31 12:20:27.042722 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:27.042722 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:27.042722 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:27.042722 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:27.042722 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:27.042722 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:27.042722 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:27.042722 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:27.042722 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:27.042722 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:27.042722 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:27.042722 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:27.043533 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:27.043533 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:27.043533 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:27.043533 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:27.043533 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 31 12:20:27.043533 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 31 12:20:27.043533 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 31 12:20:27.043533 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 31 12:20:27.043533 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 31 12:20:27.043533 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 31 12:20:27.043533 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 31 12:20:27.043533 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:20:27.043533 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 151/301] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Oct 31 12:20:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:20:27.049471 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dc6241-b169-4e9f-a162-5f00f285abd6 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:27.096798 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff2bce5-29b7-4a15-9138-75d4e5c11411 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:27.097121 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff2bce5-29b7-4a15-9138-75d4e5c11411 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:27.097357 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff2bce5-29b7-4a15-9138-75d4e5c11411 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:27.098881 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 152/302] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:20:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 31 12:20:27.105261 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5571028-fc25-476f-97da-333ff802414f None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:27.105540 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5571028-fc25-476f-97da-333ff802414f None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:27.105811 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5571028-fc25-476f-97da-333ff802414f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:27.131862 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-c5571028-fc25-476f-97da-333ff802414f None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:20:27.133045 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-c5571028-fc25-476f-97da-333ff802414f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:20:27.214015 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dc6241-b169-4e9f-a162-5f00f285abd6 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:27.226229 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 153/303] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:20:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 31 12:20:27.226905 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5571028-fc25-476f-97da-333ff802414f None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:27.227697 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:27.240771 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:27.262629 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a606f4-6240-4b3e-b432-a192c36b6447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:27.263723 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a606f4-6240-4b3e-b432-a192c36b6447 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:27.264016 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a606f4-6240-4b3e-b432-a192c36b6447 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:27.264287 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a606f4-6240-4b3e-b432-a192c36b6447 None admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:27.265228 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a606f4-6240-4b3e-b432-a192c36b6447 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:27.265530 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a606f4-6240-4b3e-b432-a192c36b6447 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:27.267459 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a606f4-6240-4b3e-b432-a192c36b6447 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:27.280003 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 151/304] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Oct 31 12:20:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:27.287149 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a606f4-6240-4b3e-b432-a192c36b6447 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:27.356645 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5571028-fc25-476f-97da-333ff802414f None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:27.406259 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d19078e-7e8a-464b-834c-6b81313a4f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:27.406919 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d19078e-7e8a-464b-834c-6b81313a4f1b None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:27.407217 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d19078e-7e8a-464b-834c-6b81313a4f1b None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:27.407492 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d19078e-7e8a-464b-834c-6b81313a4f1b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:27.420189 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a606f4-6240-4b3e-b432-a192c36b6447 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:27.420848 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d19078e-7e8a-464b-834c-6b81313a4f1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 31 12:20:27.427070 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-6d19078e-7e8a-464b-834c-6b81313a4f1b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 31 12:20:27.427070 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:27.427070 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:27.427070 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:27.427070 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:27.427070 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:27.427070 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:27.427070 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:27.427070 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:27.427070 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:27.427070 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:27.427070 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:27.427070 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 31 12:20:27.427070 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 31 12:20:27.427070 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 31 12:20:27.427070 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 31 12:20:27.427992 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 31 12:20:27.427992 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 31 12:20:27.427992 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 31 12:20:27.427992 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 31 12:20:27.427992 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:27.427992 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:27.427992 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:27.427992 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:27.427992 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:27.427992 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:27.427992 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:27.427992 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:27.427992 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:27.427992 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:27.427992 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:27.427992 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:27.428758 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:27.428758 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:27.428758 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:27.428758 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:27.428758 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 31 12:20:27.428758 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 31 12:20:27.428758 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 31 12:20:27.428758 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 31 12:20:27.428758 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 31 12:20:27.428758 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 31 12:20:27.428758 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 31 12:20:27.428758 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:20:27.428758 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 154/305] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Oct 31 12:20:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:20:27.435902 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d19078e-7e8a-464b-834c-6b81313a4f1b None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:27.483138 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b202270b-5223-436d-81e2-f33f7fe8a060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:27.483767 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b202270b-5223-436d-81e2-f33f7fe8a060 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:27.484044 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b202270b-5223-436d-81e2-f33f7fe8a060 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:27.484320 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b202270b-5223-436d-81e2-f33f7fe8a060 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:27.569384 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:27.570243 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:27.616939 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d19078e-7e8a-464b-834c-6b81313a4f1b None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:27.625730 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b202270b-5223-436d-81e2-f33f7fe8a060 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:27.627200 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b202270b-5223-436d-81e2-f33f7fe8a060 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:27.662657 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583e5baa-6505-4205-8aac-ed5a4dda2b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:27.663303 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583e5baa-6505-4205-8aac-ed5a4dda2b89 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:27.663582 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583e5baa-6505-4205-8aac-ed5a4dda2b89 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:27.663854 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583e5baa-6505-4205-8aac-ed5a4dda2b89 None admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:27.664515 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583e5baa-6505-4205-8aac-ed5a4dda2b89 None admin] RBAC: Adding query filter params (name=member) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:27.664815 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583e5baa-6505-4205-8aac-ed5a4dda2b89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:27.666428 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583e5baa-6505-4205-8aac-ed5a4dda2b89 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:27.677121 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 155/306] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Oct 31 12:20:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 31 12:20:27.678015 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583e5baa-6505-4205-8aac-ed5a4dda2b89 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:27.766546 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 152/307] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Oct 31 12:20:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 31 12:20:27.767747 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b202270b-5223-436d-81e2-f33f7fe8a060 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:27.819818 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583e5baa-6505-4205-8aac-ed5a4dda2b89 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:27.857914 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:27.858322 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:27.866460 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd60e83c-b0cd-4ea0-83dd-88940a0ab99f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:27.867985 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd60e83c-b0cd-4ea0-83dd-88940a0ab99f None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:27.868409 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd60e83c-b0cd-4ea0-83dd-88940a0ab99f None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:27.868782 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd60e83c-b0cd-4ea0-83dd-88940a0ab99f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:27.886233 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd60e83c-b0cd-4ea0-83dd-88940a0ab99f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 31 12:20:27.894391 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-bd60e83c-b0cd-4ea0-83dd-88940a0ab99f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 31 12:20:27.894391 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:27.894391 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:27.894391 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:27.894391 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:27.894391 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:27.894391 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:27.894391 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:27.894391 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:27.894391 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:27.894391 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:27.894391 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:27.894391 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 31 12:20:27.894391 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 31 12:20:27.894391 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 31 12:20:27.894391 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 31 12:20:27.895378 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 31 12:20:27.895378 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 31 12:20:27.895378 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 31 12:20:27.895378 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 31 12:20:27.895378 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:27.895378 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:27.895378 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:27.895378 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:27.895378 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:27.895378 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:27.895378 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:27.895378 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:27.895378 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:27.895378 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:27.895378 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:27.895378 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:27.896431 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:27.896431 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:27.896431 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:27.896431 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:27.896431 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 31 12:20:27.896431 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 31 12:20:27.896431 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 31 12:20:27.896431 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 31 12:20:27.896431 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 31 12:20:27.896431 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 31 12:20:27.896431 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 31 12:20:27.896431 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:20:27.897183 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b202270b-5223-436d-81e2-f33f7fe8a060 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:27.898466 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 156/308] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Oct 31 12:20:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:20:27.899419 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd60e83c-b0cd-4ea0-83dd-88940a0ab99f None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:27.949334 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c428bde3-52c1-46e3-89aa-e99fba16be4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:27.949954 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c428bde3-52c1-46e3-89aa-e99fba16be4a None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:27.950231 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c428bde3-52c1-46e3-89aa-e99fba16be4a None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:27.950965 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c428bde3-52c1-46e3-89aa-e99fba16be4a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:28.032089 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd60e83c-b0cd-4ea0-83dd-88940a0ab99f None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:28.054034 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c428bde3-52c1-46e3-89aa-e99fba16be4a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:28.055755 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c428bde3-52c1-46e3-89aa-e99fba16be4a None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:28.080219 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e08e536-6c98-4b49-b8b5-58cb78e0e9f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:28.080399 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e08e536-6c98-4b49-b8b5-58cb78e0e9f1 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:28.080605 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e08e536-6c98-4b49-b8b5-58cb78e0e9f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:28.080851 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e08e536-6c98-4b49-b8b5-58cb78e0e9f1 None admin] PATH_INFO: `/v3/projects/9bc6131ae0634994bd48138d77878576` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:28.085198 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e08e536-6c98-4b49-b8b5-58cb78e0e9f1 None admin] RBAC: Authorizing `identity:get_project(project_id=9bc6131ae0634994bd48138d77878576)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:28.085198 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e08e536-6c98-4b49-b8b5-58cb78e0e9f1 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:28.085932 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 157/309] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Oct 31 12:20:27 2022] GET /identity/v3/projects/9bc6131ae0634994bd48138d77878576 => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:28.086542 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e08e536-6c98-4b49-b8b5-58cb78e0e9f1 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:28.153291 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:28.165299 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:28.195922 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 153/310] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Oct 31 12:20:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 31 12:20:28.197321 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592ada30-7722-4b2b-a790-c534f3002953 None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:28.197768 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592ada30-7722-4b2b-a790-c534f3002953 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:28.198185 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592ada30-7722-4b2b-a790-c534f3002953 None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:28.203789 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 154/311] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:20:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:20:28.204638 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592ada30-7722-4b2b-a790-c534f3002953 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:28.221338 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e08e536-6c98-4b49-b8b5-58cb78e0e9f1 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:28.280754 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f156cd7-ece2-42f7-bfa7-70ebc6123392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:28.281374 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f156cd7-ece2-42f7-bfa7-70ebc6123392 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:28.281659 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f156cd7-ece2-42f7-bfa7-70ebc6123392 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:28.281929 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f156cd7-ece2-42f7-bfa7-70ebc6123392 None admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:28.282686 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f156cd7-ece2-42f7-bfa7-70ebc6123392 None admin] RBAC: Adding query filter params (name=member) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:28.282898 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f156cd7-ece2-42f7-bfa7-70ebc6123392 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:28.284311 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f156cd7-ece2-42f7-bfa7-70ebc6123392 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:28.299953 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 158/312] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Oct 31 12:20:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:28.301623 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f156cd7-ece2-42f7-bfa7-70ebc6123392 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:28.327826 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592ada30-7722-4b2b-a790-c534f3002953 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:28.368973 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec303fd-d13c-4620-b1d8-832d6a94d241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:28.370928 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec303fd-d13c-4620-b1d8-832d6a94d241 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:28.371241 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec303fd-d13c-4620-b1d8-832d6a94d241 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:28.371684 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec303fd-d13c-4620-b1d8-832d6a94d241 None admin] PATH_INFO: `/v3/groups/0ebd842d7a254cfcb86cfa52b46e4826` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:28.373515 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec303fd-d13c-4620-b1d8-832d6a94d241 None admin] RBAC: Authorizing `identity:get_group(group_id=0ebd842d7a254cfcb86cfa52b46e4826)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:28.375105 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec303fd-d13c-4620-b1d8-832d6a94d241 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:28.377499 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 155/313] 10.222.0.15 () {56 vars in 1180 bytes} [Mon Oct 31 12:20:28 2022] GET /identity/v3/groups/0ebd842d7a254cfcb86cfa52b46e4826 => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:28.378832 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f91e14-778b-4bdc-b7be-5f11fc2b6c96 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:28.379116 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f91e14-778b-4bdc-b7be-5f11fc2b6c96 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:28.379732 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f91e14-778b-4bdc-b7be-5f11fc2b6c96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:28.407507 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-f0f91e14-778b-4bdc-b7be-5f11fc2b6c96 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:20:28.408708 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-f0f91e14-778b-4bdc-b7be-5f11fc2b6c96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:20:28.424742 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f156cd7-ece2-42f7-bfa7-70ebc6123392 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:28.452643 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:28.455933 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:28.480907 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6e2486-7789-48f3-87b1-c52225c32d01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:28.481888 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6e2486-7789-48f3-87b1-c52225c32d01 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:28.482164 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6e2486-7789-48f3-87b1-c52225c32d01 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:28.482438 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6e2486-7789-48f3-87b1-c52225c32d01 None admin] PATH_INFO: `/v3/projects/9bc6131ae0634994bd48138d77878576` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:28.483716 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6e2486-7789-48f3-87b1-c52225c32d01 None admin] RBAC: Authorizing `identity:get_project(project_id=9bc6131ae0634994bd48138d77878576)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:28.485719 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6e2486-7789-48f3-87b1-c52225c32d01 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:28.487142 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 159/314] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Oct 31 12:20:28 2022] GET /identity/v3/projects/9bc6131ae0634994bd48138d77878576 => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:28.489415 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6e2486-7789-48f3-87b1-c52225c32d01 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:28.513154 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 156/315] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:20:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 31 12:20:28.513835 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f91e14-778b-4bdc-b7be-5f11fc2b6c96 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:28.613567 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6e2486-7789-48f3-87b1-c52225c32d01 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:28.654191 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f91e14-778b-4bdc-b7be-5f11fc2b6c96 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:28.663136 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57f82ae-1595-48e4-8096-fb2a6c62180c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:28.663789 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57f82ae-1595-48e4-8096-fb2a6c62180c None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:28.664069 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57f82ae-1595-48e4-8096-fb2a6c62180c None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:28.664343 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57f82ae-1595-48e4-8096-fb2a6c62180c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:28.694739 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e849b2-602c-4f05-9670-40b60c402892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:28.696761 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e849b2-602c-4f05-9670-40b60c402892 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:28.697259 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e849b2-602c-4f05-9670-40b60c402892 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:28.698468 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e849b2-602c-4f05-9670-40b60c402892 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:28.700140 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e849b2-602c-4f05-9670-40b60c402892 None admin] RBAC: Adding query filter params (group.id=0ebd842d7a254cfcb86cfa52b46e4826, role.id=65728d8bb0e546788426da322c34baba, scope.project.id=9bc6131ae0634994bd48138d77878576) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:28.700451 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e849b2-602c-4f05-9670-40b60c402892 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:28.701907 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e849b2-602c-4f05-9670-40b60c402892 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:28.709351 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 157/316] 10.222.0.15 () {56 vars in 1376 bytes} [Mon Oct 31 12:20:28 2022] GET /identity/v3/role_assignments?group.id=0ebd842d7a254cfcb86cfa52b46e4826&scope.project.id=9bc6131ae0634994bd48138d77878576&role.id=65728d8bb0e546788426da322c34baba => generated 611 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:28.710145 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e849b2-602c-4f05-9670-40b60c402892 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:28.744610 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:28.755438 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:28.764105 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57f82ae-1595-48e4-8096-fb2a6c62180c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:28.766175 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57f82ae-1595-48e4-8096-fb2a6c62180c None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:28.868008 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e849b2-602c-4f05-9670-40b60c402892 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:28.912051 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4874360-7ee6-45be-831f-71b1d7292734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:28.912051 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4874360-7ee6-45be-831f-71b1d7292734 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:28.912051 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4874360-7ee6-45be-831f-71b1d7292734 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:28.912502 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4874360-7ee6-45be-831f-71b1d7292734 None admin] PATH_INFO: `/v3/groups/0ebd842d7a254cfcb86cfa52b46e4826` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:28.916843 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4874360-7ee6-45be-831f-71b1d7292734 None admin] RBAC: Authorizing `identity:get_group(group_id=0ebd842d7a254cfcb86cfa52b46e4826)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:28.916843 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4874360-7ee6-45be-831f-71b1d7292734 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:28.918057 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 158/317] 10.222.0.15 () {56 vars in 1180 bytes} [Mon Oct 31 12:20:28 2022] GET /identity/v3/groups/0ebd842d7a254cfcb86cfa52b46e4826 => generated 226 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:28.919042 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b531e899-e622-45f9-84f7-062b0efba453 None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:28.919344 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b531e899-e622-45f9-84f7-062b0efba453 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:28.919616 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b531e899-e622-45f9-84f7-062b0efba453 None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:28.920817 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 159/318] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:20:28 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:20:28.921500 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b531e899-e622-45f9-84f7-062b0efba453 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:28.923664 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 160/319] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Oct 31 12:20:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 31 12:20:28.925143 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57f82ae-1595-48e4-8096-fb2a6c62180c None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:29.036061 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:29.037974 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:29.043793 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b531e899-e622-45f9-84f7-062b0efba453 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:29.051420 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57f82ae-1595-48e4-8096-fb2a6c62180c None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:29.086579 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450cc5e2-18cd-4b2e-a899-57096674cd03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:29.087277 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450cc5e2-18cd-4b2e-a899-57096674cd03 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:29.087574 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450cc5e2-18cd-4b2e-a899-57096674cd03 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:29.088626 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450cc5e2-18cd-4b2e-a899-57096674cd03 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:29.091852 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab8fc4d-f9c2-4249-ac9e-0e08d63f2433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:29.092004 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab8fc4d-f9c2-4249-ac9e-0e08d63f2433 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:29.092255 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab8fc4d-f9c2-4249-ac9e-0e08d63f2433 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:29.092529 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab8fc4d-f9c2-4249-ac9e-0e08d63f2433 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:29.093250 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab8fc4d-f9c2-4249-ac9e-0e08d63f2433 None admin] RBAC: Adding query filter params (group.id=0ebd842d7a254cfcb86cfa52b46e4826, role.id=ca1a5695f8bd499aa0b8cedaeecd2cc6, scope.project.id=9bc6131ae0634994bd48138d77878576) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:29.093552 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab8fc4d-f9c2-4249-ac9e-0e08d63f2433 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:29.094977 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab8fc4d-f9c2-4249-ac9e-0e08d63f2433 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:29.101973 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450cc5e2-18cd-4b2e-a899-57096674cd03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 31 12:20:29.102939 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 161/320] 10.222.0.15 () {56 vars in 1376 bytes} [Mon Oct 31 12:20:28 2022] GET /identity/v3/role_assignments?group.id=0ebd842d7a254cfcb86cfa52b46e4826&scope.project.id=9bc6131ae0634994bd48138d77878576&role.id=ca1a5695f8bd499aa0b8cedaeecd2cc6 => generated 611 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:29.104926 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacacdde-56ee-4c4f-b2a1-30339749822e None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:29.105277 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacacdde-56ee-4c4f-b2a1-30339749822e None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:29.105619 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacacdde-56ee-4c4f-b2a1-30339749822e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:29.106954 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-450cc5e2-18cd-4b2e-a899-57096674cd03 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 31 12:20:29.106954 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:29.106954 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:29.106954 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:29.106954 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:29.106954 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:29.106954 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:29.106954 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:29.106954 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:29.106954 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:29.106954 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:29.106954 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:29.106954 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 31 12:20:29.106954 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 31 12:20:29.106954 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 31 12:20:29.106954 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 31 12:20:29.107975 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 31 12:20:29.107975 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 31 12:20:29.107975 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 31 12:20:29.107975 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 31 12:20:29.107975 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:29.107975 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:29.107975 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:29.107975 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:29.107975 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:29.107975 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:29.107975 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:29.107975 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:29.107975 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:29.107975 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:29.107975 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:29.107975 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:29.108795 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:29.108795 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:29.108795 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:29.108795 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:29.108795 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 31 12:20:29.108795 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 31 12:20:29.108795 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 31 12:20:29.108795 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 31 12:20:29.108795 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 31 12:20:29.108795 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 31 12:20:29.108795 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 31 12:20:29.108795 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:20:29.108795 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 160/321] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Oct 31 12:20:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:20:29.109436 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450cc5e2-18cd-4b2e-a899-57096674cd03 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:29.131554 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-aacacdde-56ee-4c4f-b2a1-30339749822e None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:20:29.132757 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-aacacdde-56ee-4c4f-b2a1-30339749822e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:20:29.224629 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 162/322] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:20:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 31 12:20:29.225498 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacacdde-56ee-4c4f-b2a1-30339749822e None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:29.271375 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450cc5e2-18cd-4b2e-a899-57096674cd03 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:29.313291 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae590c8f-f01a-4aad-8780-c3b8af6c4181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:29.313928 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae590c8f-f01a-4aad-8780-c3b8af6c4181 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:29.314319 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae590c8f-f01a-4aad-8780-c3b8af6c4181 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:29.314591 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae590c8f-f01a-4aad-8780-c3b8af6c4181 None admin] PATH_INFO: `/v3/projects/574ce97a765a42729f79bf108a26330a` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:29.316613 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae590c8f-f01a-4aad-8780-c3b8af6c4181 None admin] RBAC: Authorizing `identity:get_project(project_id=574ce97a765a42729f79bf108a26330a)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:29.318033 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae590c8f-f01a-4aad-8780-c3b8af6c4181 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:29.319710 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 161/323] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Oct 31 12:20:29 2022] GET /identity/v3/projects/574ce97a765a42729f79bf108a26330a => generated 302 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:29.325619 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae590c8f-f01a-4aad-8780-c3b8af6c4181 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:29.349994 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:29.365168 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:29.378754 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacacdde-56ee-4c4f-b2a1-30339749822e None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:29.427280 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0de14b-63da-4e74-9992-d34005ebeb0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:29.428161 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0de14b-63da-4e74-9992-d34005ebeb0f None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:29.428427 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0de14b-63da-4e74-9992-d34005ebeb0f None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:29.429175 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0de14b-63da-4e74-9992-d34005ebeb0f None admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:29.429846 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0de14b-63da-4e74-9992-d34005ebeb0f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:29.430292 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0de14b-63da-4e74-9992-d34005ebeb0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:29.431841 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0de14b-63da-4e74-9992-d34005ebeb0f None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:29.443258 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 163/324] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Oct 31 12:20:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:29.449826 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0de14b-63da-4e74-9992-d34005ebeb0f None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:29.457947 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae590c8f-f01a-4aad-8780-c3b8af6c4181 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:29.495640 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa820c36-fdaf-4bb3-aa20-00a084e58e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:29.496598 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa820c36-fdaf-4bb3-aa20-00a084e58e64 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:29.497067 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa820c36-fdaf-4bb3-aa20-00a084e58e64 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:29.497523 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa820c36-fdaf-4bb3-aa20-00a084e58e64 None admin] PATH_INFO: `/v3/groups/0ebd842d7a254cfcb86cfa52b46e4826` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:29.498831 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa820c36-fdaf-4bb3-aa20-00a084e58e64 None admin] RBAC: Authorizing `identity:get_group(group_id=0ebd842d7a254cfcb86cfa52b46e4826)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:29.501008 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa820c36-fdaf-4bb3-aa20-00a084e58e64 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:29.502827 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 162/325] 10.222.0.15 () {56 vars in 1180 bytes} [Mon Oct 31 12:20:29 2022] GET /identity/v3/groups/0ebd842d7a254cfcb86cfa52b46e4826 => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 31 12:20:29.508638 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa820c36-fdaf-4bb3-aa20-00a084e58e64 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:29.578494 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0de14b-63da-4e74-9992-d34005ebeb0f None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:29.613929 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0ba5fc-828c-49d3-b83c-a2473109cf4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:29.614543 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0ba5fc-828c-49d3-b83c-a2473109cf4c None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:29.614816 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0ba5fc-828c-49d3-b83c-a2473109cf4c None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:29.615079 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0ba5fc-828c-49d3-b83c-a2473109cf4c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:29.626373 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:29.631453 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:29.633241 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa820c36-fdaf-4bb3-aa20-00a084e58e64 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:29.681438 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4b1277-277d-4f23-be86-0586308a9255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:29.682253 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4b1277-277d-4f23-be86-0586308a9255 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:29.682605 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4b1277-277d-4f23-be86-0586308a9255 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:29.682878 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4b1277-277d-4f23-be86-0586308a9255 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:29.683699 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4b1277-277d-4f23-be86-0586308a9255 None admin] RBAC: Adding query filter params (group.id=0ebd842d7a254cfcb86cfa52b46e4826, role.id=ca1a5695f8bd499aa0b8cedaeecd2cc6, scope.project.id=574ce97a765a42729f79bf108a26330a) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:29.684003 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4b1277-277d-4f23-be86-0586308a9255 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:29.685422 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4b1277-277d-4f23-be86-0586308a9255 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:29.692387 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 163/326] 10.222.0.15 () {56 vars in 1376 bytes} [Mon Oct 31 12:20:29 2022] GET /identity/v3/role_assignments?group.id=0ebd842d7a254cfcb86cfa52b46e4826&scope.project.id=574ce97a765a42729f79bf108a26330a&role.id=ca1a5695f8bd499aa0b8cedaeecd2cc6 => generated 611 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 31 12:20:29.692533 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4b1277-277d-4f23-be86-0586308a9255 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:29.704464 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0ba5fc-828c-49d3-b83c-a2473109cf4c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:29.706426 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0ba5fc-828c-49d3-b83c-a2473109cf4c None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:29.823233 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4b1277-277d-4f23-be86-0586308a9255 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:29.842076 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 164/327] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Oct 31 12:20:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 31 12:20:29.843281 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0ba5fc-828c-49d3-b83c-a2473109cf4c None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:29.876359 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0de010-746c-49b6-9a90-a32b794b0960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:29.876999 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0de010-746c-49b6-9a90-a32b794b0960 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:29.877273 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0de010-746c-49b6-9a90-a32b794b0960 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:29.877540 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0de010-746c-49b6-9a90-a32b794b0960 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:29.888726 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0de010-746c-49b6-9a90-a32b794b0960 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 31 12:20:29.890887 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:29.896134 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-8f0de010-746c-49b6-9a90-a32b794b0960 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 31 12:20:29.896134 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:29.896134 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:29.896134 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:29.896134 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:29.896134 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:29.896134 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:29.896134 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:29.896134 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:29.896134 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:29.896134 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:29.896134 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:29.896134 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 31 12:20:29.896134 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 31 12:20:29.896134 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 31 12:20:29.896134 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 31 12:20:29.897011 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 31 12:20:29.897011 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 31 12:20:29.897011 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 31 12:20:29.897011 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 31 12:20:29.897011 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:29.897011 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:29.897011 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:29.897011 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:29.897011 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:29.897011 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:29.897011 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:29.897011 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:29.897011 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:29.897011 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:29.897011 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:29.897011 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:29.897808 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:29.897808 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:29.897808 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:29.897808 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:29.897808 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 31 12:20:29.897808 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 31 12:20:29.897808 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 31 12:20:29.897808 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 31 12:20:29.897808 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 31 12:20:29.897808 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 31 12:20:29.897808 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 31 12:20:29.897808 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:20:29.897808 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 164/328] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Oct 31 12:20:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:20:29.904301 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0de010-746c-49b6-9a90-a32b794b0960 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:29.928287 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:29.949610 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0ba5fc-828c-49d3-b83c-a2473109cf4c None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:29.991977 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aff9b7-244d-438c-bf38-3c152f517d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:29.992787 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aff9b7-244d-438c-bf38-3c152f517d91 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:29.993066 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aff9b7-244d-438c-bf38-3c152f517d91 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:29.994127 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aff9b7-244d-438c-bf38-3c152f517d91 None admin] PATH_INFO: `/v3/projects/574ce97a765a42729f79bf108a26330a` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:29.996073 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02aff9b7-244d-438c-bf38-3c152f517d91 None admin] RBAC: Authorizing `identity:get_project(project_id=574ce97a765a42729f79bf108a26330a)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:29.997465 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02aff9b7-244d-438c-bf38-3c152f517d91 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:30.001606 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 165/329] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Oct 31 12:20:29 2022] GET /identity/v3/projects/574ce97a765a42729f79bf108a26330a => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:30.006549 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aff9b7-244d-438c-bf38-3c152f517d91 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:30.032274 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0de010-746c-49b6-9a90-a32b794b0960 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:30.074654 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e1bf97-d38a-4612-945a-50d0bada6ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:30.075216 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e1bf97-d38a-4612-945a-50d0bada6ed1 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:30.075496 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e1bf97-d38a-4612-945a-50d0bada6ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:30.075867 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e1bf97-d38a-4612-945a-50d0bada6ed1 None admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:30.076414 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e1bf97-d38a-4612-945a-50d0bada6ed1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:30.076716 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e1bf97-d38a-4612-945a-50d0bada6ed1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:30.078082 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e1bf97-d38a-4612-945a-50d0bada6ed1 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:30.090680 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 165/330] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Oct 31 12:20:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:30.096724 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e1bf97-d38a-4612-945a-50d0bada6ed1 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:30.124287 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aff9b7-244d-438c-bf38-3c152f517d91 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:30.137918 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:30.159999 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39b03eb-cad1-4cc1-87d0-a616cfd02b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:30.160624 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39b03eb-cad1-4cc1-87d0-a616cfd02b45 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:30.160901 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39b03eb-cad1-4cc1-87d0-a616cfd02b45 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:30.161166 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39b03eb-cad1-4cc1-87d0-a616cfd02b45 None admin] PATH_INFO: `/v3/groups/0ebd842d7a254cfcb86cfa52b46e4826` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:30.162260 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39b03eb-cad1-4cc1-87d0-a616cfd02b45 None admin] RBAC: Authorizing `identity:get_group(group_id=0ebd842d7a254cfcb86cfa52b46e4826)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:30.163742 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39b03eb-cad1-4cc1-87d0-a616cfd02b45 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:30.165297 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 166/331] 10.222.0.15 () {56 vars in 1180 bytes} [Mon Oct 31 12:20:30 2022] GET /identity/v3/groups/0ebd842d7a254cfcb86cfa52b46e4826 => generated 226 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 31 12:20:30.171175 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39b03eb-cad1-4cc1-87d0-a616cfd02b45 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:30.179662 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:30.198883 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e1bf97-d38a-4612-945a-50d0bada6ed1 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:30.232562 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bbc60c-76b9-44ca-8ee5-aead37202d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:30.233160 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bbc60c-76b9-44ca-8ee5-aead37202d81 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:30.233432 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bbc60c-76b9-44ca-8ee5-aead37202d81 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:30.233698 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bbc60c-76b9-44ca-8ee5-aead37202d81 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:30.270782 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39b03eb-cad1-4cc1-87d0-a616cfd02b45 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:30.300670 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bbc60c-76b9-44ca-8ee5-aead37202d81 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:30.302209 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bbc60c-76b9-44ca-8ee5-aead37202d81 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:30.304324 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46342908-0b54-4b32-a7d1-b7853cfbd4f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:30.305693 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46342908-0b54-4b32-a7d1-b7853cfbd4f8 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:30.305976 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46342908-0b54-4b32-a7d1-b7853cfbd4f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:30.306247 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46342908-0b54-4b32-a7d1-b7853cfbd4f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:30.306985 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46342908-0b54-4b32-a7d1-b7853cfbd4f8 None admin] RBAC: Adding query filter params (group.id=0ebd842d7a254cfcb86cfa52b46e4826, role.id=65728d8bb0e546788426da322c34baba, scope.project.id=574ce97a765a42729f79bf108a26330a) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:30.307470 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46342908-0b54-4b32-a7d1-b7853cfbd4f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:30.308869 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46342908-0b54-4b32-a7d1-b7853cfbd4f8 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:30.315736 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 167/332] 10.222.0.15 () {56 vars in 1376 bytes} [Mon Oct 31 12:20:30 2022] GET /identity/v3/role_assignments?group.id=0ebd842d7a254cfcb86cfa52b46e4826&scope.project.id=574ce97a765a42729f79bf108a26330a&role.id=65728d8bb0e546788426da322c34baba => generated 611 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 31 12:20:30.391080 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:30.412325 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 166/333] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Oct 31 12:20:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 31 12:20:30.418391 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46342908-0b54-4b32-a7d1-b7853cfbd4f8 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:30.467231 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:30.526521 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46342908-0b54-4b32-a7d1-b7853cfbd4f8 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:30.562364 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38aa15a-c0e0-416f-9f59-f9b015ac3c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:30.563014 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38aa15a-c0e0-416f-9f59-f9b015ac3c7f None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:30.563302 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38aa15a-c0e0-416f-9f59-f9b015ac3c7f None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:30.563574 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38aa15a-c0e0-416f-9f59-f9b015ac3c7f None admin] PATH_INFO: `/v3/projects/20cfeec04f5e4394b2ab5ceaa7d9c63a` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:30.564677 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38aa15a-c0e0-416f-9f59-f9b015ac3c7f None admin] RBAC: Authorizing `identity:get_project(project_id=20cfeec04f5e4394b2ab5ceaa7d9c63a)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:30.566104 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38aa15a-c0e0-416f-9f59-f9b015ac3c7f None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:30.567917 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 168/334] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Oct 31 12:20:30 2022] GET /identity/v3/projects/20cfeec04f5e4394b2ab5ceaa7d9c63a => generated 344 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:30.573487 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bbc60c-76b9-44ca-8ee5-aead37202d81 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:30.674271 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bbc60c-76b9-44ca-8ee5-aead37202d81 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:30.707883 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbf3c80-6f20-4390-99c3-91658a50329d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:30.708496 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbf3c80-6f20-4390-99c3-91658a50329d None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:30.708767 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbf3c80-6f20-4390-99c3-91658a50329d None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:30.709038 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbf3c80-6f20-4390-99c3-91658a50329d None admin] PATH_INFO: `/v3/groups/c7526800273345768325c7449c0812c0` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:30.710112 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbf3c80-6f20-4390-99c3-91658a50329d None admin] RBAC: Authorizing `identity:get_group(group_id=c7526800273345768325c7449c0812c0)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:30.711568 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbf3c80-6f20-4390-99c3-91658a50329d None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:30.713975 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 167/335] 10.222.0.15 () {56 vars in 1180 bytes} [Mon Oct 31 12:20:30 2022] GET /identity/v3/groups/c7526800273345768325c7449c0812c0 => generated 229 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:30.719882 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38aa15a-c0e0-416f-9f59-f9b015ac3c7f None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:30.814595 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38aa15a-c0e0-416f-9f59-f9b015ac3c7f None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:30.826669 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:30.847829 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea2bd41-8734-463e-931c-ed08405835b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:30.848416 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea2bd41-8734-463e-931c-ed08405835b5 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:30.848689 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea2bd41-8734-463e-931c-ed08405835b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:30.848951 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea2bd41-8734-463e-931c-ed08405835b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:30.849652 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea2bd41-8734-463e-931c-ed08405835b5 None admin] RBAC: Adding query filter params (group.id=c7526800273345768325c7449c0812c0, role.id=176c07b4fe3b4537bdfeb4be21b379c5, scope.project.id=20cfeec04f5e4394b2ab5ceaa7d9c63a) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:30.849952 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea2bd41-8734-463e-931c-ed08405835b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:30.851377 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea2bd41-8734-463e-931c-ed08405835b5 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:30.857127 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 169/336] 10.222.0.15 () {56 vars in 1376 bytes} [Mon Oct 31 12:20:30 2022] GET /identity/v3/role_assignments?group.id=c7526800273345768325c7449c0812c0&scope.project.id=20cfeec04f5e4394b2ab5ceaa7d9c63a&role.id=176c07b4fe3b4537bdfeb4be21b379c5 => generated 611 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:33.655247 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1137754e-c582-4ca7-b97b-d4eafca9b0a2 None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:33.655770 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1137754e-c582-4ca7-b97b-d4eafca9b0a2 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:33.655770 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1137754e-c582-4ca7-b97b-d4eafca9b0a2 None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:33.657118 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 168/337] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:20:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 31 12:20:33.662459 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b714fcd4-01c2-4487-9a1e-ba71b9132763 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:33.662854 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b714fcd4-01c2-4487-9a1e-ba71b9132763 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:33.663036 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b714fcd4-01c2-4487-9a1e-ba71b9132763 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:33.685741 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-b714fcd4-01c2-4487-9a1e-ba71b9132763 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:20:33.686958 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-b714fcd4-01c2-4487-9a1e-ba71b9132763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:20:33.750101 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcac92ff-2c5a-4fc2-9c7b-0d4ded7b9c97 None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:33.750424 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcac92ff-2c5a-4fc2-9c7b-0d4ded7b9c97 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:33.750656 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcac92ff-2c5a-4fc2-9c7b-0d4ded7b9c97 None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:33.752106 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 169/338] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:20:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 31 12:20:33.763509 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ab528d-99ec-48ed-b446-14e81a699f27 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:33.763802 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ab528d-99ec-48ed-b446-14e81a699f27 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:33.764061 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ab528d-99ec-48ed-b446-14e81a699f27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:33.783991 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-b1ab528d-99ec-48ed-b446-14e81a699f27 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:20:33.785506 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-b1ab528d-99ec-48ed-b446-14e81a699f27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:20:33.793189 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 170/339] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:20:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 31 12:20:33.845754 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:33.867831 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 170/340] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:20:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Oct 31 12:20:33.935425 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e12f4d6-d0ff-4051-a5b6-262c5d7b45f7 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:33.935607 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e12f4d6-d0ff-4051-a5b6-262c5d7b45f7 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:33.935891 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e12f4d6-d0ff-4051-a5b6-262c5d7b45f7 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:33.937251 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 171/341] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:20:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 31 12:20:33.942693 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40109de5-7165-487a-8ce3-ace92f817612 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:33.942978 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40109de5-7165-487a-8ce3-ace92f817612 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:33.943278 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40109de5-7165-487a-8ce3-ace92f817612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:33.971887 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-40109de5-7165-487a-8ce3-ace92f817612 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:20:33.973058 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-40109de5-7165-487a-8ce3-ace92f817612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:20:34.046194 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 171/342] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:20:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 31 12:20:34.245722 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e12f4d6-d0ff-4051-a5b6-262c5d7b45f7 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:34.300064 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ad4bb8-8525-4905-bd3d-f3b9f4a071d2 None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:34.300398 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ad4bb8-8525-4905-bd3d-f3b9f4a071d2 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:34.300608 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ad4bb8-8525-4905-bd3d-f3b9f4a071d2 None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:34.302094 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 172/343] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:20:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 31 12:20:34.304000 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ad4bb8-8525-4905-bd3d-f3b9f4a071d2 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:34.359562 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e12f4d6-d0ff-4051-a5b6-262c5d7b45f7 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:34.368235 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:34.404281 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2968b3ac-2142-41f4-8467-8e5c0f583dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:34.405022 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2968b3ac-2142-41f4-8467-8e5c0f583dcb None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:34.405299 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2968b3ac-2142-41f4-8467-8e5c0f583dcb None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:34.405564 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2968b3ac-2142-41f4-8467-8e5c0f583dcb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:34.414846 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ad4bb8-8525-4905-bd3d-f3b9f4a071d2 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:34.416110 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2968b3ac-2142-41f4-8467-8e5c0f583dcb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:34.417777 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2968b3ac-2142-41f4-8467-8e5c0f583dcb None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:34.425781 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:34.434620 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-2968b3ac-2142-41f4-8467-8e5c0f583dcb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:20:34.434620 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:34.434620 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:34.434620 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:34.434620 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:34.434620 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:34.434620 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:34.434620 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:34.434620 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:34.434620 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:34.434620 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:34.434620 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:34.434620 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 31 12:20:34.434620 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 31 12:20:34.434620 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 31 12:20:34.434620 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 31 12:20:34.435683 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:34.435683 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:34.435683 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:34.435683 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:34.435683 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:34.435683 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:34.435683 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:34.435683 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:34.435683 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:34.435683 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:34.435683 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:34.435683 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:34.435683 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:34.435683 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:34.435683 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:34.435683 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:34.435683 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 31 12:20:34.441571 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 31 12:20:34.441571 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:20:34.441571 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:20:34.441571 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 172/344] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Oct 31 12:20:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:20:34.441571 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8033ee-87fd-485d-804f-5980cf49d3f3 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:34.441571 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8033ee-87fd-485d-804f-5980cf49d3f3 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:34.441571 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8033ee-87fd-485d-804f-5980cf49d3f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:34.454436 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6698c36-29e9-4e7b-b149-7635ab1f93d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:34.455085 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6698c36-29e9-4e7b-b149-7635ab1f93d5 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:34.455386 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6698c36-29e9-4e7b-b149-7635ab1f93d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:34.455648 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6698c36-29e9-4e7b-b149-7635ab1f93d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:34.457599 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-2a8033ee-87fd-485d-804f-5980cf49d3f3 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:20:34.458768 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-2a8033ee-87fd-485d-804f-5980cf49d3f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:20:34.466094 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6698c36-29e9-4e7b-b149-7635ab1f93d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:34.467543 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6698c36-29e9-4e7b-b149-7635ab1f93d5 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:34.485421 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-b6698c36-29e9-4e7b-b149-7635ab1f93d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:20:34.485421 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:34.485421 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:34.485421 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:34.485421 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:34.485421 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:34.485421 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:34.485421 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:34.485421 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:34.485421 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:34.485421 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:34.485421 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:34.485421 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 31 12:20:34.485421 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 31 12:20:34.485421 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 31 12:20:34.485421 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 31 12:20:34.486339 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:34.486339 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:34.486339 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:34.486339 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:34.486339 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:34.486339 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:34.486339 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:34.486339 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:34.486339 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:34.486339 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:34.486339 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:34.486339 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:34.486339 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:34.486339 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:34.486339 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:34.486339 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:34.486339 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 31 12:20:34.487924 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 31 12:20:34.487924 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:20:34.487924 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:20:34.487924 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 173/345] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Oct 31 12:20:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:20:34.487924 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6698c36-29e9-4e7b-b149-7635ab1f93d5 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:34.540800 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 173/346] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:20:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 31 12:20:34.541673 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8033ee-87fd-485d-804f-5980cf49d3f3 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:34.610960 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6698c36-29e9-4e7b-b149-7635ab1f93d5 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:34.653580 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f6add0-108c-4fca-b3e3-ce44f78f6809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:34.654724 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f6add0-108c-4fca-b3e3-ce44f78f6809 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:34.655221 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f6add0-108c-4fca-b3e3-ce44f78f6809 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:34.655627 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f6add0-108c-4fca-b3e3-ce44f78f6809 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:34.657138 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:34.671190 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f6add0-108c-4fca-b3e3-ce44f78f6809 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:34.672961 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f6add0-108c-4fca-b3e3-ce44f78f6809 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:34.673751 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8033ee-87fd-485d-804f-5980cf49d3f3 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:34.691889 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-b8f6add0-108c-4fca-b3e3-ce44f78f6809 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:20:34.691889 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:34.691889 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:34.691889 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:34.691889 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:34.691889 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:34.691889 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:34.691889 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:34.691889 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:34.691889 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:34.691889 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:34.691889 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:34.691889 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 31 12:20:34.691889 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 31 12:20:34.691889 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 31 12:20:34.691889 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 31 12:20:34.693040 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:34.693040 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:34.693040 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:34.693040 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:34.693040 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:34.693040 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:34.693040 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:34.693040 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:34.693040 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:34.693040 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:34.693040 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:34.693040 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:34.693040 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:34.693040 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:34.693040 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:34.693040 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:34.693040 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 31 12:20:34.693858 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 31 12:20:34.693858 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:20:34.693858 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:20:34.696244 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 174/347] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Oct 31 12:20:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:20:34.697327 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f6add0-108c-4fca-b3e3-ce44f78f6809 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:34.717886 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:34.720249 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a5017b-a1c0-42c4-8abb-db016277f2ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:34.721166 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a5017b-a1c0-42c4-8abb-db016277f2ac None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:34.721497 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a5017b-a1c0-42c4-8abb-db016277f2ac None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:34.721835 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a5017b-a1c0-42c4-8abb-db016277f2ac None admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:34.723002 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a5017b-a1c0-42c4-8abb-db016277f2ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:34.723656 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a5017b-a1c0-42c4-8abb-db016277f2ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:34.725459 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a5017b-a1c0-42c4-8abb-db016277f2ac None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:34.742417 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 174/348] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Oct 31 12:20:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:34.744462 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a5017b-a1c0-42c4-8abb-db016277f2ac None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:34.834095 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f6add0-108c-4fca-b3e3-ce44f78f6809 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:34.868697 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a5017b-a1c0-42c4-8abb-db016277f2ac None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:34.871126 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ec9f7d-9f92-49b5-a384-4ef9f3dd2d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:34.871770 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ec9f7d-9f92-49b5-a384-4ef9f3dd2d5c None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:34.872056 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ec9f7d-9f92-49b5-a384-4ef9f3dd2d5c None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:34.872318 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ec9f7d-9f92-49b5-a384-4ef9f3dd2d5c None admin] PATH_INFO: `/v3/domains` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:34.872980 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ec9f7d-9f92-49b5-a384-4ef9f3dd2d5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:34.873547 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ec9f7d-9f92-49b5-a384-4ef9f3dd2d5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:34.875002 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ec9f7d-9f92-49b5-a384-4ef9f3dd2d5c None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:34.887689 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 175/349] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Oct 31 12:20:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:34.888603 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ec9f7d-9f92-49b5-a384-4ef9f3dd2d5c None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:34.906800 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4063eee9-0470-4aed-83ff-cf515c82237b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:34.907446 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4063eee9-0470-4aed-83ff-cf515c82237b None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:34.907725 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4063eee9-0470-4aed-83ff-cf515c82237b None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:34.908002 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4063eee9-0470-4aed-83ff-cf515c82237b None admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:34.908633 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4063eee9-0470-4aed-83ff-cf515c82237b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:34.908933 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4063eee9-0470-4aed-83ff-cf515c82237b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:34.910283 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4063eee9-0470-4aed-83ff-cf515c82237b None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:34.922621 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 175/350] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Oct 31 12:20:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:34.924089 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4063eee9-0470-4aed-83ff-cf515c82237b None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:34.935714 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:34.962718 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:34.988515 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ec9f7d-9f92-49b5-a384-4ef9f3dd2d5c None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:35.023645 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8323e6-6810-49f1-a659-b54283c060ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:35.024500 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8323e6-6810-49f1-a659-b54283c060ee None admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:35.024927 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8323e6-6810-49f1-a659-b54283c060ee None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:35.025327 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8323e6-6810-49f1-a659-b54283c060ee None admin] PATH_INFO: `/v3/users` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:35.026854 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8323e6-6810-49f1-a659-b54283c060ee None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:35.028456 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8323e6-6810-49f1-a659-b54283c060ee None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:35.028860 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4063eee9-0470-4aed-83ff-cf515c82237b None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:35.050533 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 176/351] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Oct 31 12:20:34 2022] POST /identity/v3/users => generated 246 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:20:35.051875 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049edd31-73a3-4222-8161-a39fe08fa9f6 None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:35.052152 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049edd31-73a3-4222-8161-a39fe08fa9f6 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:35.052420 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049edd31-73a3-4222-8161-a39fe08fa9f6 None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:35.053630 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 177/352] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:20:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:20:35.054319 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049edd31-73a3-4222-8161-a39fe08fa9f6 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:35.082182 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d37d5b4-8626-4d94-bc61-597f4f153b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:35.083959 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d37d5b4-8626-4d94-bc61-597f4f153b45 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:35.084485 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d37d5b4-8626-4d94-bc61-597f4f153b45 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:35.085161 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d37d5b4-8626-4d94-bc61-597f4f153b45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:35.099286 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d37d5b4-8626-4d94-bc61-597f4f153b45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:35.101155 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d37d5b4-8626-4d94-bc61-597f4f153b45 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:35.120303 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-3d37d5b4-8626-4d94-bc61-597f4f153b45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:20:35.120303 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:35.120303 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:35.120303 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:35.120303 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:35.120303 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:35.120303 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:35.120303 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:35.120303 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:35.120303 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:35.120303 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:35.120303 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:35.120303 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 31 12:20:35.120303 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 31 12:20:35.120303 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 31 12:20:35.120303 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 31 12:20:35.121267 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:35.121267 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:35.121267 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:35.121267 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:35.121267 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:35.121267 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:35.121267 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:35.121267 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:35.121267 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:35.121267 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:35.121267 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:35.121267 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:35.121267 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:35.121267 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:35.121267 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:35.121267 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:35.121267 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 31 12:20:35.122091 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 31 12:20:35.122091 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:20:35.122091 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:20:35.122091 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 176/353] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Oct 31 12:20:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:20:35.122299 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d37d5b4-8626-4d94-bc61-597f4f153b45 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:35.175756 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049edd31-73a3-4222-8161-a39fe08fa9f6 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:35.195732 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:35.210931 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:35.221114 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d47144e-27f9-4ebb-b0d7-08751b7007ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:35.221726 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d47144e-27f9-4ebb-b0d7-08751b7007ce None admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:35.222001 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d47144e-27f9-4ebb-b0d7-08751b7007ce None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:35.222269 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d47144e-27f9-4ebb-b0d7-08751b7007ce None admin] PATH_INFO: `/v3/users` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:35.222910 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d47144e-27f9-4ebb-b0d7-08751b7007ce None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:35.224362 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d47144e-27f9-4ebb-b0d7-08751b7007ce None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:35.244833 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d37d5b4-8626-4d94-bc61-597f4f153b45 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:35.250216 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 178/354] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Oct 31 12:20:35 2022] POST /identity/v3/users => generated 248 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:20:35.251616 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16c6d08-c0e0-48a8-9c15-5409ed6c79cf None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:35.251902 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16c6d08-c0e0-48a8-9c15-5409ed6c79cf None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:35.252173 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16c6d08-c0e0-48a8-9c15-5409ed6c79cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:35.274771 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-b16c6d08-c0e0-48a8-9c15-5409ed6c79cf None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:20:35.276672 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-b16c6d08-c0e0-48a8-9c15-5409ed6c79cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:20:35.292060 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8d9928-f276-4b42-b9a8-fbd05f31dcb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:35.293261 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8d9928-f276-4b42-b9a8-fbd05f31dcb9 None admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:35.294095 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8d9928-f276-4b42-b9a8-fbd05f31dcb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:35.294374 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8d9928-f276-4b42-b9a8-fbd05f31dcb9 None admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:35.295487 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8d9928-f276-4b42-b9a8-fbd05f31dcb9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:35.297022 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8d9928-f276-4b42-b9a8-fbd05f31dcb9 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:35.327262 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 177/355] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Oct 31 12:20:35 2022] POST /identity/v3/users => generated 248 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:20:35.328824 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8d9928-f276-4b42-b9a8-fbd05f31dcb9 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:35.370009 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 179/356] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:20:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 31 12:20:35.455947 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8d9928-f276-4b42-b9a8-fbd05f31dcb9 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:35.474769 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:35.495057 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5ecce8-42f5-4047-b262-3b5d27723829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:35.496461 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5ecce8-42f5-4047-b262-3b5d27723829 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:35.496812 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5ecce8-42f5-4047-b262-3b5d27723829 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:35.497157 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5ecce8-42f5-4047-b262-3b5d27723829 None admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:35.497878 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5ecce8-42f5-4047-b262-3b5d27723829 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:35.498419 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5ecce8-42f5-4047-b262-3b5d27723829 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:35.500019 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5ecce8-42f5-4047-b262-3b5d27723829 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:35.514402 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 178/357] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Oct 31 12:20:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:35.520004 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16c6d08-c0e0-48a8-9c15-5409ed6c79cf None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:35.647099 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16c6d08-c0e0-48a8-9c15-5409ed6c79cf None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:35.664690 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:35.685203 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36bf062-4e6f-4f6d-8791-776e62735382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:35.685809 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36bf062-4e6f-4f6d-8791-776e62735382 None admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:35.686086 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36bf062-4e6f-4f6d-8791-776e62735382 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:35.686356 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36bf062-4e6f-4f6d-8791-776e62735382 None admin] PATH_INFO: `/v3/users` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:35.687001 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36bf062-4e6f-4f6d-8791-776e62735382 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:35.688525 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36bf062-4e6f-4f6d-8791-776e62735382 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:35.709739 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5ecce8-42f5-4047-b262-3b5d27723829 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:35.714915 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 180/358] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Oct 31 12:20:35 2022] POST /identity/v3/users => generated 249 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:20:35.831200 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5ecce8-42f5-4047-b262-3b5d27723829 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:35.871573 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef105581-ff35-4cdb-899d-4191780d56b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:35.872198 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef105581-ff35-4cdb-899d-4191780d56b7 None admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:35.872472 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef105581-ff35-4cdb-899d-4191780d56b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:35.872742 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef105581-ff35-4cdb-899d-4191780d56b7 None admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:35.873418 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef105581-ff35-4cdb-899d-4191780d56b7 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:35.874792 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef105581-ff35-4cdb-899d-4191780d56b7 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:35.885825 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.sql.core [None req-ef105581-ff35-4cdb-899d-4191780d56b7 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 31 12:20:35.885825 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 31 12:20:35.885825 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [parameters: {'id': '1b0b2f6453cf494da1fddf6c10fc11df', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 31 12:20:35.885825 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81491) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 31 12:20:35.893110 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-ef105581-ff35-4cdb-899d-4191780d56b7 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 31 12:20:35.893110 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:35.893110 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 31 12:20:35.893110 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 31 12:20:35.893110 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 31 12:20:35.893110 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 31 12:20:35.893110 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 31 12:20:35.893110 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application result = self._query(query) Oct 31 12:20:35.893110 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 31 12:20:35.893110 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application conn.query(q) Oct 31 12:20:35.893110 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 31 12:20:35.893110 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 31 12:20:35.893110 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 31 12:20:35.893110 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application result.read() Oct 31 12:20:35.893110 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 31 12:20:35.894278 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 31 12:20:35.894278 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 31 12:20:35.894278 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 31 12:20:35.894278 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 31 12:20:35.894278 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 31 12:20:35.894278 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 31 12:20:35.894278 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 31 12:20:35.894278 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 31 12:20:35.894278 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:20:35.894278 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 31 12:20:35.894278 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:20:35.894278 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:35.894278 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 31 12:20:35.894278 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 31 12:20:35.894278 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Oct 31 12:20:35.894278 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return ref.to_dict() Oct 31 12:20:35.894278 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 31 12:20:35.894278 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application next(self.gen) Oct 31 12:20:35.895698 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Oct 31 12:20:35.895698 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application yield resource Oct 31 12:20:35.895698 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 31 12:20:35.895698 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application next(self.gen) Oct 31 12:20:35.895698 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Oct 31 12:20:35.895698 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self.session.rollback() Oct 31 12:20:35.895698 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 31 12:20:35.895698 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self.force_reraise() Oct 31 12:20:35.895698 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 31 12:20:35.895698 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise self.value Oct 31 12:20:35.895698 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Oct 31 12:20:35.895698 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 31 12:20:35.895698 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Oct 31 12:20:35.895698 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application session.commit() Oct 31 12:20:35.895698 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 31 12:20:35.895698 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 31 12:20:35.895698 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 31 12:20:35.896716 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self._prepare_impl() Oct 31 12:20:35.896716 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 31 12:20:35.896716 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self.session.flush() Oct 31 12:20:35.896716 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 31 12:20:35.896716 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self._flush(objects) Oct 31 12:20:35.896716 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 31 12:20:35.896716 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 31 12:20:35.896716 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 31 12:20:35.896716 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application compat.raise_( Oct 31 12:20:35.896716 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 31 12:20:35.896716 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception Oct 31 12:20:35.896716 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 31 12:20:35.896716 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application flush_context.execute() Oct 31 12:20:35.896716 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 31 12:20:35.896716 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rec.execute(self) Oct 31 12:20:35.896716 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 31 12:20:35.896716 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 31 12:20:35.897588 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 31 12:20:35.897588 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 31 12:20:35.897588 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 31 12:20:35.897588 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 31 12:20:35.897588 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 31 12:20:35.897588 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 31 12:20:35.897588 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 31 12:20:35.897588 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 31 12:20:35.897588 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 31 12:20:35.897588 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 31 12:20:35.897588 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 31 12:20:35.897588 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 31 12:20:35.897588 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 31 12:20:35.897588 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 31 12:20:35.897588 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 31 12:20:35.897588 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception Oct 31 12:20:35.898376 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 31 12:20:35.898376 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 31 12:20:35.898376 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 31 12:20:35.898376 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 31 12:20:35.898376 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 31 12:20:35.898376 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application result = self._query(query) Oct 31 12:20:35.898376 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 31 12:20:35.898376 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application conn.query(q) Oct 31 12:20:35.898376 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 31 12:20:35.898376 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 31 12:20:35.898376 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 31 12:20:35.898376 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application result.read() Oct 31 12:20:35.898376 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 31 12:20:35.898376 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 31 12:20:35.898376 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 31 12:20:35.898376 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 31 12:20:35.900072 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 31 12:20:35.900072 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 31 12:20:35.900072 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 31 12:20:35.900072 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 31 12:20:35.900072 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 31 12:20:35.900072 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 31 12:20:35.900072 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [parameters: {'id': '1b0b2f6453cf494da1fddf6c10fc11df', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 31 12:20:35.900072 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 31 12:20:35.900072 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:20:35.900072 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 31 12:20:35.900072 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:20:35.900072 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:35.900072 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:35.900072 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:35.900072 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:35.900072 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:35.901763 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:35.901763 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:35.901763 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:35.901763 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:35.901763 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:35.901763 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:35.901763 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Oct 31 12:20:35.901763 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Oct 31 12:20:35.901763 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:35.901763 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:35.901763 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Oct 31 12:20:35.901763 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Oct 31 12:20:35.901763 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 31 12:20:35.901763 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 31 12:20:35.901763 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 31 12:20:35.901763 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:20:35.902705 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 179/359] 10.222.0.15 () {60 vars in 1131 bytes} [Mon Oct 31 12:20:35 2022] POST /identity/v3/roles => generated 144 bytes in 186 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 31 12:20:35.902786 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36bf062-4e6f-4f6d-8791-776e62735382 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:36.054218 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36bf062-4e6f-4f6d-8791-776e62735382 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:36.111613 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6926a09a-1db5-401e-bf94-05ca351b73b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:36.112463 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6926a09a-1db5-401e-bf94-05ca351b73b4 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:36.112824 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6926a09a-1db5-401e-bf94-05ca351b73b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:36.113216 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6926a09a-1db5-401e-bf94-05ca351b73b4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:36.127300 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6926a09a-1db5-401e-bf94-05ca351b73b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 31 12:20:36.132958 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-6926a09a-1db5-401e-bf94-05ca351b73b4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 31 12:20:36.132958 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:36.132958 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:36.132958 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:36.132958 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:36.132958 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:36.132958 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:36.132958 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:36.132958 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:36.132958 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:36.132958 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:36.132958 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:36.132958 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 31 12:20:36.132958 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 31 12:20:36.132958 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 31 12:20:36.132958 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 31 12:20:36.134058 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 31 12:20:36.134058 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 31 12:20:36.134058 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 31 12:20:36.134058 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 31 12:20:36.134058 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:36.134058 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:36.134058 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:36.134058 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:36.134058 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:36.134058 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:36.134058 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:36.134058 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:36.134058 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:36.134058 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:36.134058 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:36.134058 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:36.135871 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:36.135871 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:36.135871 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:36.135871 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:36.135871 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 31 12:20:36.135871 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 31 12:20:36.135871 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 31 12:20:36.135871 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 31 12:20:36.135871 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 31 12:20:36.135871 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 31 12:20:36.135871 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 31 12:20:36.135871 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:20:36.137437 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 181/360] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Oct 31 12:20:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:20:36.141787 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef105581-ff35-4cdb-899d-4191780d56b7 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:36.202473 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:36.269479 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef105581-ff35-4cdb-899d-4191780d56b7 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:36.306893 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7eccf7-3f72-4968-964d-60ccefeda286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:36.307541 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7eccf7-3f72-4968-964d-60ccefeda286 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:36.307820 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7eccf7-3f72-4968-964d-60ccefeda286 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:36.308089 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7eccf7-3f72-4968-964d-60ccefeda286 None admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:36.308753 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7eccf7-3f72-4968-964d-60ccefeda286 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:36.309055 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7eccf7-3f72-4968-964d-60ccefeda286 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:36.310423 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7eccf7-3f72-4968-964d-60ccefeda286 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:36.319216 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 180/361] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Oct 31 12:20:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:37.698461 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be01be00-6659-43ad-a36b-10d8cc699ccb None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:37.699113 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be01be00-6659-43ad-a36b-10d8cc699ccb None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:37.699113 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be01be00-6659-43ad-a36b-10d8cc699ccb None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:37.700642 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 182/362] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:20:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:20:37.712504 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e49c0d-45ae-417f-97c7-9a8e78607e07 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:37.712942 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e49c0d-45ae-417f-97c7-9a8e78607e07 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:37.713201 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e49c0d-45ae-417f-97c7-9a8e78607e07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:37.735705 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-02e49c0d-45ae-417f-97c7-9a8e78607e07 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:20:37.736892 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-02e49c0d-45ae-417f-97c7-9a8e78607e07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:20:37.802701 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d471ae-6d61-4be7-b9fa-8e0d06aafcb5 None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:37.803038 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d471ae-6d61-4be7-b9fa-8e0d06aafcb5 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:37.803271 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d471ae-6d61-4be7-b9fa-8e0d06aafcb5 None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:37.804623 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 183/363] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:20:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 31 12:20:37.809736 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88690e45-bfec-45cc-8a22-4ebc1bb405e5 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:37.810017 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88690e45-bfec-45cc-8a22-4ebc1bb405e5 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:37.810293 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88690e45-bfec-45cc-8a22-4ebc1bb405e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:37.832792 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-88690e45-bfec-45cc-8a22-4ebc1bb405e5 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:20:37.834653 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-88690e45-bfec-45cc-8a22-4ebc1bb405e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:20:37.849723 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 181/364] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:20:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 31 12:20:37.896141 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:37.923315 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 184/365] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:20:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 31 12:20:38.206665 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e49c0d-45ae-417f-97c7-9a8e78607e07 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:38.217823 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88690e45-bfec-45cc-8a22-4ebc1bb405e5 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:38.245618 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:38.374612 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e49c0d-45ae-417f-97c7-9a8e78607e07 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:38.380576 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88690e45-bfec-45cc-8a22-4ebc1bb405e5 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:38.441901 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9817798a-5c38-476e-a0fb-7e30173ca55a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:38.442952 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9817798a-5c38-476e-a0fb-7e30173ca55a None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:38.443469 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9817798a-5c38-476e-a0fb-7e30173ca55a None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:38.444817 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9817798a-5c38-476e-a0fb-7e30173ca55a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:38.465552 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febeb12f-9b68-40fd-8066-77eb95582771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:38.466196 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febeb12f-9b68-40fd-8066-77eb95582771 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:38.466469 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febeb12f-9b68-40fd-8066-77eb95582771 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:38.466731 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febeb12f-9b68-40fd-8066-77eb95582771 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:38.467896 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9817798a-5c38-476e-a0fb-7e30173ca55a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 31 12:20:38.472752 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-9817798a-5c38-476e-a0fb-7e30173ca55a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 31 12:20:38.472752 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:38.472752 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:38.472752 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:38.472752 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:38.472752 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:38.472752 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:38.472752 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:38.472752 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:38.472752 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:38.472752 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:38.472752 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:38.472752 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 31 12:20:38.472752 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 31 12:20:38.472752 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 31 12:20:38.472752 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 31 12:20:38.473588 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 31 12:20:38.473588 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 31 12:20:38.473588 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 31 12:20:38.473588 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 31 12:20:38.473588 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:38.473588 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:38.473588 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:38.473588 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:38.473588 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:38.473588 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:38.473588 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:38.473588 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:38.473588 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:38.473588 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:38.473588 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:38.473588 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:38.474361 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:38.474361 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:38.474361 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:38.474361 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:38.474361 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 31 12:20:38.474361 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 31 12:20:38.474361 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 31 12:20:38.474361 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 31 12:20:38.474361 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 31 12:20:38.474361 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 31 12:20:38.474361 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 31 12:20:38.474361 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:20:38.474361 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 182/366] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Oct 31 12:20:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:20:38.475551 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a15ecdc-2293-43b5-85d9-c2e5f648f989 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:38.475655 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a15ecdc-2293-43b5-85d9-c2e5f648f989 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:38.475924 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a15ecdc-2293-43b5-85d9-c2e5f648f989 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:38.477138 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 183/367] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:20:38 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:20:38.483394 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a15ecdc-2293-43b5-85d9-c2e5f648f989 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:38.486621 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febeb12f-9b68-40fd-8066-77eb95582771 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 31 12:20:38.491459 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-febeb12f-9b68-40fd-8066-77eb95582771 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 31 12:20:38.491459 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:38.491459 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:38.491459 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:38.491459 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:38.491459 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:38.491459 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:38.491459 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:38.491459 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:38.491459 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:38.491459 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:38.491459 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:38.491459 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 31 12:20:38.491459 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 31 12:20:38.491459 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 31 12:20:38.491459 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 31 12:20:38.492364 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 31 12:20:38.492364 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 31 12:20:38.492364 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 31 12:20:38.492364 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 31 12:20:38.492364 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:38.492364 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:38.492364 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:38.492364 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:38.492364 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:38.492364 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:38.492364 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:38.492364 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:38.492364 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:38.492364 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:38.492364 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:38.492364 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:38.493184 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:38.493184 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:38.493184 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:38.493184 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:38.493184 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 31 12:20:38.493184 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 31 12:20:38.493184 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 31 12:20:38.493184 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 31 12:20:38.493184 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 31 12:20:38.493184 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 31 12:20:38.493184 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 31 12:20:38.493184 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:20:38.493184 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 185/368] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Oct 31 12:20:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 31 12:20:38.493822 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62578209-6069-4753-9c62-1439ae80b2bc None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:38.499027 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62578209-6069-4753-9c62-1439ae80b2bc None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:38.499027 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62578209-6069-4753-9c62-1439ae80b2bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:38.517216 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-62578209-6069-4753-9c62-1439ae80b2bc None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:20:38.518558 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-62578209-6069-4753-9c62-1439ae80b2bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:20:38.537661 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:38.595332 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 186/369] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:20:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 31 12:20:38.595332 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62578209-6069-4753-9c62-1439ae80b2bc None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:38.605323 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:38.616446 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a15ecdc-2293-43b5-85d9-c2e5f648f989 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:38.653586 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbfa03b-7451-4842-bc60-cb82ad1f7424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:38.654417 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbfa03b-7451-4842-bc60-cb82ad1f7424 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:38.654852 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbfa03b-7451-4842-bc60-cb82ad1f7424 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:38.655296 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbfa03b-7451-4842-bc60-cb82ad1f7424 None admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:38.656114 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbfa03b-7451-4842-bc60-cb82ad1f7424 None admin] RBAC: Adding query filter params (name=service) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:38.656560 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbfa03b-7451-4842-bc60-cb82ad1f7424 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:38.658096 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbfa03b-7451-4842-bc60-cb82ad1f7424 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:38.667864 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 184/370] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Oct 31 12:20:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:38.669371 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd50bf5-4c71-4427-8b7d-ed6eb0132bf0 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:38.669848 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd50bf5-4c71-4427-8b7d-ed6eb0132bf0 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:38.670342 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd50bf5-4c71-4427-8b7d-ed6eb0132bf0 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:38.671755 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 185/371] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:20:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:20:38.676203 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd50bf5-4c71-4427-8b7d-ed6eb0132bf0 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:38.738771 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62578209-6069-4753-9c62-1439ae80b2bc None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:38.784243 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3730ab4a-5d0a-4d89-a10c-76b54c6689a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:38.785260 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3730ab4a-5d0a-4d89-a10c-76b54c6689a2 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:38.785543 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3730ab4a-5d0a-4d89-a10c-76b54c6689a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:38.785814 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3730ab4a-5d0a-4d89-a10c-76b54c6689a2 None admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:38.786761 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3730ab4a-5d0a-4d89-a10c-76b54c6689a2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:38.787064 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3730ab4a-5d0a-4d89-a10c-76b54c6689a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:38.788718 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3730ab4a-5d0a-4d89-a10c-76b54c6689a2 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:38.800043 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 187/372] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Oct 31 12:20:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:38.801314 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fe3edc-56c5-4242-a155-f5233a170327 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:38.801598 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fe3edc-56c5-4242-a155-f5233a170327 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:38.801867 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fe3edc-56c5-4242-a155-f5233a170327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:38.813836 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd50bf5-4c71-4427-8b7d-ed6eb0132bf0 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:38.825613 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-18fe3edc-56c5-4242-a155-f5233a170327 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:20:38.827254 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-18fe3edc-56c5-4242-a155-f5233a170327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:20:38.840674 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:38.859332 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5074ed-cd10-4f4e-bc59-d62b36715b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:38.860378 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5074ed-cd10-4f4e-bc59-d62b36715b40 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:38.860858 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5074ed-cd10-4f4e-bc59-d62b36715b40 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:38.861341 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5074ed-cd10-4f4e-bc59-d62b36715b40 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:38.877764 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5074ed-cd10-4f4e-bc59-d62b36715b40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:38.879257 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5074ed-cd10-4f4e-bc59-d62b36715b40 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:38.894678 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-bb5074ed-cd10-4f4e-bc59-d62b36715b40 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:20:38.894678 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:38.894678 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:38.894678 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:38.894678 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:38.894678 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:38.894678 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:38.894678 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:38.894678 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:38.894678 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:38.894678 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:38.894678 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:38.894678 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 31 12:20:38.894678 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 31 12:20:38.894678 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 31 12:20:38.894678 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 31 12:20:38.901556 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:38.901556 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:38.901556 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:38.901556 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:38.901556 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:38.901556 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:38.901556 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:38.901556 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:38.901556 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:38.901556 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:38.901556 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:38.901556 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:38.901556 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:38.901556 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:38.901556 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:38.901556 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:38.901556 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 31 12:20:38.902444 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 31 12:20:38.902444 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:20:38.902444 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:20:38.902444 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 186/373] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Oct 31 12:20:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:20:38.902444 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5074ed-cd10-4f4e-bc59-d62b36715b40 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:38.909506 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:38.914010 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 188/374] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:20:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 31 12:20:38.914749 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fe3edc-56c5-4242-a155-f5233a170327 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:39.019094 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5074ed-cd10-4f4e-bc59-d62b36715b40 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:39.029081 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fe3edc-56c5-4242-a155-f5233a170327 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:39.054864 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e764540a-ba16-4871-9007-6c18d0ec8765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:39.055778 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e764540a-ba16-4871-9007-6c18d0ec8765 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:39.055960 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e764540a-ba16-4871-9007-6c18d0ec8765 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:39.056216 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e764540a-ba16-4871-9007-6c18d0ec8765 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:39.067628 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cba4f3c-5d5e-4c90-9e04-bc0dcdf29e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:39.068600 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cba4f3c-5d5e-4c90-9e04-bc0dcdf29e62 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:39.069080 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cba4f3c-5d5e-4c90-9e04-bc0dcdf29e62 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:39.069505 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e764540a-ba16-4871-9007-6c18d0ec8765 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:39.069775 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cba4f3c-5d5e-4c90-9e04-bc0dcdf29e62 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:39.070977 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e764540a-ba16-4871-9007-6c18d0ec8765 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:39.082725 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cba4f3c-5d5e-4c90-9e04-bc0dcdf29e62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 31 12:20:39.088726 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-3cba4f3c-5d5e-4c90-9e04-bc0dcdf29e62 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 31 12:20:39.088726 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:39.088726 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:39.088726 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:39.088726 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:39.088726 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:39.088726 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:39.088726 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:39.088726 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:39.088726 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:39.088726 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:39.088726 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:39.088726 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 31 12:20:39.088726 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 31 12:20:39.088726 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 31 12:20:39.088726 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 31 12:20:39.089872 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 31 12:20:39.089872 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 31 12:20:39.089872 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 31 12:20:39.089872 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 31 12:20:39.089872 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:39.089872 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:39.089872 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:39.089872 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:39.089872 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:39.089872 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:39.089872 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:39.089872 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:39.089872 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:39.089872 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:39.089872 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:39.089872 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:39.090676 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:39.090676 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:39.090676 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:39.090676 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:39.090676 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 31 12:20:39.090676 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 31 12:20:39.090676 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 31 12:20:39.090676 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 31 12:20:39.090676 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 31 12:20:39.090676 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 31 12:20:39.090676 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 31 12:20:39.090676 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:20:39.091346 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-e764540a-ba16-4871-9007-6c18d0ec8765 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:20:39.091346 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:39.091346 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:39.091346 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:39.091346 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:39.091346 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:39.091346 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:39.091346 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:39.091346 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:39.091346 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:39.091346 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:39.091346 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:39.091346 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 31 12:20:39.091346 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 31 12:20:39.091346 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 31 12:20:39.091346 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 31 12:20:39.092139 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:39.092139 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:39.092139 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:39.092139 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:39.092139 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:39.092139 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:39.092139 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:39.092139 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:39.092139 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:39.092139 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:39.092139 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:39.092139 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:39.092139 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:39.092139 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:39.092139 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:39.092139 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:39.092139 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 31 12:20:39.092994 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 31 12:20:39.092994 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:20:39.092994 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:20:39.092994 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 187/375] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Oct 31 12:20:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:20:39.096128 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 189/376] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Oct 31 12:20:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:20:39.096637 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e764540a-ba16-4871-9007-6c18d0ec8765 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:39.097654 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cba4f3c-5d5e-4c90-9e04-bc0dcdf29e62 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:39.107508 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:39.165674 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:39.211642 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e764540a-ba16-4871-9007-6c18d0ec8765 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:39.217436 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cba4f3c-5d5e-4c90-9e04-bc0dcdf29e62 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:39.243730 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8472f560-e29e-4153-b2eb-799e7b8d7ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:39.244958 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8472f560-e29e-4153-b2eb-799e7b8d7ac8 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:39.245235 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8472f560-e29e-4153-b2eb-799e7b8d7ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:39.245499 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8472f560-e29e-4153-b2eb-799e7b8d7ac8 None admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:39.246131 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8472f560-e29e-4153-b2eb-799e7b8d7ac8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:39.246423 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8472f560-e29e-4153-b2eb-799e7b8d7ac8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:39.247785 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8472f560-e29e-4153-b2eb-799e7b8d7ac8 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:39.253313 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75431733-e502-409b-80dc-1d3f5e3a232e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:39.253908 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75431733-e502-409b-80dc-1d3f5e3a232e None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:39.254198 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75431733-e502-409b-80dc-1d3f5e3a232e None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:39.254458 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75431733-e502-409b-80dc-1d3f5e3a232e None admin] PATH_INFO: `/v3/domains` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:39.255090 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75431733-e502-409b-80dc-1d3f5e3a232e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:39.255456 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75431733-e502-409b-80dc-1d3f5e3a232e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:39.256800 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75431733-e502-409b-80dc-1d3f5e3a232e None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:39.264152 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 188/377] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Oct 31 12:20:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:39.264152 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8472f560-e29e-4153-b2eb-799e7b8d7ac8 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:39.273159 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 190/378] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Oct 31 12:20:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:39.273969 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75431733-e502-409b-80dc-1d3f5e3a232e None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:39.338730 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:39.366906 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8472f560-e29e-4153-b2eb-799e7b8d7ac8 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:39.376541 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75431733-e502-409b-80dc-1d3f5e3a232e None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:39.384424 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:39.402859 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d7d7c5-218c-47fb-a8f6-454ef605904e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:39.403498 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d7d7c5-218c-47fb-a8f6-454ef605904e None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:39.403862 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d7d7c5-218c-47fb-a8f6-454ef605904e None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:39.404134 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d7d7c5-218c-47fb-a8f6-454ef605904e None admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:39.404870 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d7d7c5-218c-47fb-a8f6-454ef605904e None admin] RBAC: Adding query filter params (name=service) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:39.405172 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d7d7c5-218c-47fb-a8f6-454ef605904e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:39.406615 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d7d7c5-218c-47fb-a8f6-454ef605904e None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:39.409165 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85887916-95b0-42af-a9b8-6767cb2d3a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:39.409788 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85887916-95b0-42af-a9b8-6767cb2d3a66 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:39.410065 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85887916-95b0-42af-a9b8-6767cb2d3a66 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:39.410335 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85887916-95b0-42af-a9b8-6767cb2d3a66 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:39.414611 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 189/379] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Oct 31 12:20:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:39.420238 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38d2925-0aae-44ff-bbbb-e0734a5453dd None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:39.420238 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38d2925-0aae-44ff-bbbb-e0734a5453dd None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:39.420238 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38d2925-0aae-44ff-bbbb-e0734a5453dd None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:39.420238 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 190/380] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:20:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:20:39.420238 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38d2925-0aae-44ff-bbbb-e0734a5453dd None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:39.421701 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85887916-95b0-42af-a9b8-6767cb2d3a66 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 31 12:20:39.426970 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-85887916-95b0-42af-a9b8-6767cb2d3a66 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 31 12:20:39.426970 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:39.426970 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:39.426970 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:39.426970 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:39.426970 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:39.426970 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:39.426970 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:39.426970 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:39.426970 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:39.426970 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:39.426970 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:39.426970 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 31 12:20:39.426970 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 31 12:20:39.426970 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 31 12:20:39.426970 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 31 12:20:39.428072 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 31 12:20:39.428072 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 31 12:20:39.428072 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 31 12:20:39.428072 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 31 12:20:39.428072 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:39.428072 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:39.428072 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:39.428072 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:39.428072 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:39.428072 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:39.428072 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:39.428072 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:39.428072 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:39.428072 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:39.428072 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:39.428072 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:39.428850 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:39.428850 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:39.428850 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:39.428850 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:39.428850 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 31 12:20:39.428850 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 31 12:20:39.428850 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 31 12:20:39.428850 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 31 12:20:39.428850 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 31 12:20:39.428850 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 31 12:20:39.428850 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 31 12:20:39.428850 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:20:39.428850 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 191/381] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Oct 31 12:20:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:20:39.430372 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85887916-95b0-42af-a9b8-6767cb2d3a66 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:39.521859 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38d2925-0aae-44ff-bbbb-e0734a5453dd None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:39.530703 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85887916-95b0-42af-a9b8-6767cb2d3a66 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:39.553757 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:39.555552 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69137718-da38-45de-8015-a70ec9dfe948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:39.556187 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69137718-da38-45de-8015-a70ec9dfe948 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:39.556466 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69137718-da38-45de-8015-a70ec9dfe948 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:39.556762 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69137718-da38-45de-8015-a70ec9dfe948 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:39.564883 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80b9f5e-9727-47a6-8d69-d2663ff87740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:39.565123 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69137718-da38-45de-8015-a70ec9dfe948 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:39.565496 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80b9f5e-9727-47a6-8d69-d2663ff87740 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:39.565775 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80b9f5e-9727-47a6-8d69-d2663ff87740 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:39.566041 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80b9f5e-9727-47a6-8d69-d2663ff87740 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:39.566540 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69137718-da38-45de-8015-a70ec9dfe948 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:39.573751 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80b9f5e-9727-47a6-8d69-d2663ff87740 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:39.575220 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80b9f5e-9727-47a6-8d69-d2663ff87740 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:39.578766 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-69137718-da38-45de-8015-a70ec9dfe948 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 31 12:20:39.578766 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:39.578766 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:39.578766 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:39.578766 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:39.578766 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:39.578766 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:39.578766 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:39.578766 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:39.578766 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:39.578766 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:39.578766 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:39.578766 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 31 12:20:39.578766 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 31 12:20:39.578766 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 31 12:20:39.578766 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 31 12:20:39.579692 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:39.579692 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:39.579692 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 31 12:20:39.579692 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 31 12:20:39.579692 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 31 12:20:39.579692 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 31 12:20:39.579692 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:39.579692 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:39.579692 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:39.579692 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:39.579692 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:39.579692 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:39.579692 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:39.579692 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:39.579692 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:39.579692 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:39.579692 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:39.579692 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:39.580607 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:39.580607 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:39.580607 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 31 12:20:39.580607 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 31 12:20:39.580607 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 31 12:20:39.580607 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 31 12:20:39.580607 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 31 12:20:39.580607 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 31 12:20:39.580607 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 31 12:20:39.580607 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:20:39.580607 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 191/382] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Oct 31 12:20:39 2022] GET /identity/v3/users/cinder => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:20:39.580607 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69137718-da38-45de-8015-a70ec9dfe948 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:39.591930 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-f80b9f5e-9727-47a6-8d69-d2663ff87740 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 31 12:20:39.591930 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:39.591930 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:39.591930 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:39.591930 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:39.591930 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:39.591930 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:39.591930 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:39.591930 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:39.591930 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:39.591930 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:39.591930 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:39.591930 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 31 12:20:39.591930 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 31 12:20:39.591930 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 31 12:20:39.591930 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 31 12:20:39.592946 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:39.592946 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:39.592946 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 31 12:20:39.592946 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 31 12:20:39.592946 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 31 12:20:39.592946 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 31 12:20:39.592946 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:39.592946 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:39.592946 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:39.592946 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:39.592946 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:39.592946 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:39.592946 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:39.592946 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:39.592946 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:39.592946 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:39.592946 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:39.592946 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:39.594336 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:39.594336 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:39.594336 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 31 12:20:39.594336 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 31 12:20:39.594336 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 31 12:20:39.594336 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 31 12:20:39.594336 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 31 12:20:39.594336 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 31 12:20:39.594336 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 31 12:20:39.594336 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:20:39.594336 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 192/383] 10.222.0.15 () {56 vars in 1093 bytes} [Mon Oct 31 12:20:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:20:39.594336 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45727f35-546c-4ab2-b70e-e91809e464ce None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:39.595087 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45727f35-546c-4ab2-b70e-e91809e464ce None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:39.595087 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45727f35-546c-4ab2-b70e-e91809e464ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:39.613439 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-45727f35-546c-4ab2-b70e-e91809e464ce None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:20:39.614629 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-45727f35-546c-4ab2-b70e-e91809e464ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:20:39.628234 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:39.679210 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 193/384] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:20:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 31 12:20:39.680005 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45727f35-546c-4ab2-b70e-e91809e464ce None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:39.685975 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69137718-da38-45de-8015-a70ec9dfe948 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:39.720237 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f75062f-d0fb-49e0-bdca-a3d504a41387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:39.720860 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f75062f-d0fb-49e0-bdca-a3d504a41387 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:39.721136 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f75062f-d0fb-49e0-bdca-a3d504a41387 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:39.721404 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f75062f-d0fb-49e0-bdca-a3d504a41387 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:39.732157 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f75062f-d0fb-49e0-bdca-a3d504a41387 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:39.733587 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f75062f-d0fb-49e0-bdca-a3d504a41387 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:39.747850 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-3f75062f-d0fb-49e0-bdca-a3d504a41387 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:20:39.747850 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:39.747850 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:39.747850 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:39.747850 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:39.747850 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:39.747850 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:39.747850 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:39.747850 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:39.747850 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:39.747850 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:39.747850 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:39.747850 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 31 12:20:39.747850 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 31 12:20:39.747850 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 31 12:20:39.747850 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 31 12:20:39.748730 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:39.748730 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:39.748730 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:39.748730 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:39.748730 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:39.748730 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:39.748730 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:39.748730 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:39.748730 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:39.748730 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:39.748730 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:39.748730 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:39.748730 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:39.748730 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:39.748730 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:39.748730 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:39.748730 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 31 12:20:39.749562 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 31 12:20:39.749562 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:20:39.749562 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:20:39.749562 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 192/385] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Oct 31 12:20:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:20:39.749777 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f75062f-d0fb-49e0-bdca-a3d504a41387 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:39.783253 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45727f35-546c-4ab2-b70e-e91809e464ce None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:39.797354 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:39.818800 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1d0f8d-a775-4dc9-b111-b201924fbaba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:39.819440 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1d0f8d-a775-4dc9-b111-b201924fbaba None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:39.819720 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1d0f8d-a775-4dc9-b111-b201924fbaba None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:39.819989 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1d0f8d-a775-4dc9-b111-b201924fbaba None admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:39.820640 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1d0f8d-a775-4dc9-b111-b201924fbaba None admin] RBAC: Adding query filter params (name=service) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:39.820939 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1d0f8d-a775-4dc9-b111-b201924fbaba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:39.822305 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1d0f8d-a775-4dc9-b111-b201924fbaba None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:39.830334 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 194/386] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Oct 31 12:20:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:39.831127 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1d0f8d-a775-4dc9-b111-b201924fbaba None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:39.849402 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:39.859493 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f75062f-d0fb-49e0-bdca-a3d504a41387 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:39.896611 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660859ba-0821-4c48-bb45-fa4005c5401b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:39.897224 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660859ba-0821-4c48-bb45-fa4005c5401b None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:39.897521 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660859ba-0821-4c48-bb45-fa4005c5401b None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:39.897761 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660859ba-0821-4c48-bb45-fa4005c5401b None admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:39.898470 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660859ba-0821-4c48-bb45-fa4005c5401b None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:39.898767 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660859ba-0821-4c48-bb45-fa4005c5401b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:39.900204 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660859ba-0821-4c48-bb45-fa4005c5401b None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:39.933797 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1d0f8d-a775-4dc9-b111-b201924fbaba None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:39.942982 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 193/387] 10.222.0.15 () {56 vars in 1137 bytes} [Mon Oct 31 12:20:39 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:39.942982 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660859ba-0821-4c48-bb45-fa4005c5401b None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:39.973855 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53b91b7-4ded-4dea-afa7-714b94480e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:39.974481 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53b91b7-4ded-4dea-afa7-714b94480e41 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:39.974765 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53b91b7-4ded-4dea-afa7-714b94480e41 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:39.975041 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53b91b7-4ded-4dea-afa7-714b94480e41 None admin] PATH_INFO: `/v3/users` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:39.975781 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53b91b7-4ded-4dea-afa7-714b94480e41 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:39.976083 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53b91b7-4ded-4dea-afa7-714b94480e41 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:39.977494 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53b91b7-4ded-4dea-afa7-714b94480e41 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:40.014120 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 195/388] 10.222.0.15 () {56 vars in 1133 bytes} [Mon Oct 31 12:20:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:40.014976 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53b91b7-4ded-4dea-afa7-714b94480e41 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:40.042867 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660859ba-0821-4c48-bb45-fa4005c5401b None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:40.052783 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:40.076431 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0e287a-5317-429a-a81d-e6d354da0912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:40.077027 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0e287a-5317-429a-a81d-e6d354da0912 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:40.077302 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0e287a-5317-429a-a81d-e6d354da0912 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:40.077574 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0e287a-5317-429a-a81d-e6d354da0912 None admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:40.078204 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0e287a-5317-429a-a81d-e6d354da0912 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:40.078498 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0e287a-5317-429a-a81d-e6d354da0912 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:40.079888 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0e287a-5317-429a-a81d-e6d354da0912 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:40.090967 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 194/389] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Oct 31 12:20:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:40.091780 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0e287a-5317-429a-a81d-e6d354da0912 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:40.104595 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:40.124979 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53b91b7-4ded-4dea-afa7-714b94480e41 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:40.158722 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e490aed-ef7d-477b-a44e-c5f3f9f98e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:40.159396 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e490aed-ef7d-477b-a44e-c5f3f9f98e6f None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:40.159662 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e490aed-ef7d-477b-a44e-c5f3f9f98e6f None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:40.159928 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e490aed-ef7d-477b-a44e-c5f3f9f98e6f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:40.170248 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e490aed-ef7d-477b-a44e-c5f3f9f98e6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:40.171752 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e490aed-ef7d-477b-a44e-c5f3f9f98e6f None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:40.186923 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-0e490aed-ef7d-477b-a44e-c5f3f9f98e6f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:20:40.186923 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:40.186923 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:40.186923 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:40.186923 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:40.186923 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:40.186923 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:40.186923 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:40.186923 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:40.186923 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:40.186923 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:40.186923 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:40.186923 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 31 12:20:40.186923 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 31 12:20:40.186923 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 31 12:20:40.186923 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 31 12:20:40.187915 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:40.187915 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:40.187915 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:40.187915 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:40.187915 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:40.187915 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:40.187915 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:40.187915 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:40.187915 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:40.187915 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:40.187915 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:40.187915 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:40.187915 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:40.187915 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:40.187915 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:40.187915 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:40.187915 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 31 12:20:40.188761 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 31 12:20:40.188761 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:20:40.188761 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:20:40.188761 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 196/390] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Oct 31 12:20:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:20:40.188761 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e490aed-ef7d-477b-a44e-c5f3f9f98e6f None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:40.195554 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0e287a-5317-429a-a81d-e6d354da0912 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:40.229572 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08c77e3-84e1-4261-aac3-a1ffbef748d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:40.230200 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08c77e3-84e1-4261-aac3-a1ffbef748d5 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:40.230478 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08c77e3-84e1-4261-aac3-a1ffbef748d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:40.230750 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08c77e3-84e1-4261-aac3-a1ffbef748d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:40.242359 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08c77e3-84e1-4261-aac3-a1ffbef748d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:40.243849 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08c77e3-84e1-4261-aac3-a1ffbef748d5 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:40.258241 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-e08c77e3-84e1-4261-aac3-a1ffbef748d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:20:40.258241 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:40.258241 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:40.258241 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:40.258241 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:40.258241 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:40.258241 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:40.258241 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:40.258241 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:40.258241 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:40.258241 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:40.258241 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:40.258241 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 31 12:20:40.258241 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 31 12:20:40.258241 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 31 12:20:40.258241 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 31 12:20:40.259129 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:40.259129 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:40.259129 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:40.259129 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:40.259129 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:40.259129 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:40.259129 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:40.259129 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:40.259129 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:40.259129 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:40.259129 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:40.259129 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:40.259129 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:40.259129 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:40.259129 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:40.259129 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:40.259129 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 31 12:20:40.260116 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 31 12:20:40.260116 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:20:40.260116 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:20:40.260116 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 195/391] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Oct 31 12:20:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:20:40.260332 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08c77e3-84e1-4261-aac3-a1ffbef748d5 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:40.280702 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:40.296619 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e490aed-ef7d-477b-a44e-c5f3f9f98e6f None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:40.320949 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:40.333066 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc383c73-ba5e-467d-a178-916a85a8aa51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:40.333691 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc383c73-ba5e-467d-a178-916a85a8aa51 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:40.333967 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc383c73-ba5e-467d-a178-916a85a8aa51 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:40.334236 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc383c73-ba5e-467d-a178-916a85a8aa51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:40.345352 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc383c73-ba5e-467d-a178-916a85a8aa51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:40.347061 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc383c73-ba5e-467d-a178-916a85a8aa51 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:40.361414 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-dc383c73-ba5e-467d-a178-916a85a8aa51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:20:40.361414 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:40.361414 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:40.361414 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:40.361414 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:40.361414 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:40.361414 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:40.361414 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:40.361414 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:40.361414 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:40.361414 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:40.361414 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:40.361414 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 31 12:20:40.361414 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 31 12:20:40.361414 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 31 12:20:40.361414 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 31 12:20:40.362342 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:40.362342 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:40.362342 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:40.362342 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:40.362342 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:40.362342 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:40.362342 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:40.362342 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:40.362342 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:40.362342 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:40.362342 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:40.362342 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:40.362342 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:40.362342 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:40.362342 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:40.362342 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:40.362342 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 31 12:20:40.363220 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 31 12:20:40.363220 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:20:40.363220 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:20:40.363387 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08c77e3-84e1-4261-aac3-a1ffbef748d5 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:40.363466 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 197/392] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Oct 31 12:20:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:20:40.363466 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc383c73-ba5e-467d-a178-916a85a8aa51 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:40.397101 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece79f48-998e-4c5c-9d59-d8abc9eb0188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:40.397727 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece79f48-998e-4c5c-9d59-d8abc9eb0188 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:40.398009 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece79f48-998e-4c5c-9d59-d8abc9eb0188 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:40.398277 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece79f48-998e-4c5c-9d59-d8abc9eb0188 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:40.410679 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece79f48-998e-4c5c-9d59-d8abc9eb0188 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:40.413066 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece79f48-998e-4c5c-9d59-d8abc9eb0188 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:40.427787 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-ece79f48-998e-4c5c-9d59-d8abc9eb0188 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:20:40.427787 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:40.427787 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:40.427787 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:40.427787 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:40.427787 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:40.427787 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:40.427787 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:40.427787 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:40.427787 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:40.427787 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:40.427787 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:40.427787 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 31 12:20:40.427787 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 31 12:20:40.427787 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 31 12:20:40.427787 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 31 12:20:40.428738 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:40.428738 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:40.428738 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:40.428738 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:40.428738 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:40.428738 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:40.428738 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:40.428738 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:40.428738 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:40.428738 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:40.428738 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:40.428738 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:40.428738 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:40.428738 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:40.428738 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:40.428738 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:40.428738 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 31 12:20:40.430043 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 31 12:20:40.430043 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:20:40.430043 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:20:40.430043 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 196/393] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Oct 31 12:20:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:20:40.430043 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece79f48-998e-4c5c-9d59-d8abc9eb0188 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:40.471256 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc383c73-ba5e-467d-a178-916a85a8aa51 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:40.504599 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ae4fd9-fe8f-458d-b88f-a8aeff538619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:40.505218 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ae4fd9-fe8f-458d-b88f-a8aeff538619 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:40.505496 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ae4fd9-fe8f-458d-b88f-a8aeff538619 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:40.505759 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ae4fd9-fe8f-458d-b88f-a8aeff538619 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:40.513036 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ae4fd9-fe8f-458d-b88f-a8aeff538619 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:40.514474 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ae4fd9-fe8f-458d-b88f-a8aeff538619 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:40.516193 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:40.527189 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-87ae4fd9-fe8f-458d-b88f-a8aeff538619 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 31 12:20:40.527189 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:40.527189 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:40.527189 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:40.527189 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:40.527189 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:40.527189 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:40.527189 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:40.527189 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:40.527189 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:40.527189 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:40.527189 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:40.527189 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 31 12:20:40.527189 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 31 12:20:40.527189 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 31 12:20:40.527189 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 31 12:20:40.528080 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:40.528080 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:40.528080 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 31 12:20:40.528080 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 31 12:20:40.528080 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 31 12:20:40.528080 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 31 12:20:40.528080 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:40.528080 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:40.528080 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:40.528080 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:40.528080 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:40.528080 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:40.528080 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:40.528080 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:40.528080 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:40.528080 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:40.528080 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:40.528080 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:40.528947 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:40.528947 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:40.528947 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 31 12:20:40.528947 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 31 12:20:40.528947 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 31 12:20:40.528947 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 31 12:20:40.528947 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 31 12:20:40.528947 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 31 12:20:40.528947 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 31 12:20:40.528947 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:20:40.528947 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 198/394] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Oct 31 12:20:40 2022] GET /identity/v3/users/glance => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:20:40.529488 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ae4fd9-fe8f-458d-b88f-a8aeff538619 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:40.530035 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece79f48-998e-4c5c-9d59-d8abc9eb0188 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:40.543596 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:40.564164 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caa6e46-7a61-45de-a97c-15f5160d043e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:40.564771 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caa6e46-7a61-45de-a97c-15f5160d043e None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:40.565051 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caa6e46-7a61-45de-a97c-15f5160d043e None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:40.565318 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caa6e46-7a61-45de-a97c-15f5160d043e None admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:40.566077 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caa6e46-7a61-45de-a97c-15f5160d043e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:40.566379 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caa6e46-7a61-45de-a97c-15f5160d043e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:40.567771 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caa6e46-7a61-45de-a97c-15f5160d043e None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:40.579494 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 197/395] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Oct 31 12:20:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:40.580284 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caa6e46-7a61-45de-a97c-15f5160d043e None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:40.633874 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ae4fd9-fe8f-458d-b88f-a8aeff538619 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:40.668452 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6b073d-14c0-4a7e-841d-d338b496c578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:40.669052 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6b073d-14c0-4a7e-841d-d338b496c578 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:40.669335 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6b073d-14c0-4a7e-841d-d338b496c578 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:40.669606 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6b073d-14c0-4a7e-841d-d338b496c578 None admin] PATH_INFO: `/v3/domains` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:40.670236 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6b073d-14c0-4a7e-841d-d338b496c578 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:40.670534 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6b073d-14c0-4a7e-841d-d338b496c578 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:40.671932 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6b073d-14c0-4a7e-841d-d338b496c578 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:40.676287 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caa6e46-7a61-45de-a97c-15f5160d043e None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:40.683638 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 199/396] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Oct 31 12:20:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:40.684260 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6b073d-14c0-4a7e-841d-d338b496c578 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:40.711907 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236b40b0-1a51-4f3a-8615-ac2a76a219aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:40.712511 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236b40b0-1a51-4f3a-8615-ac2a76a219aa None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:40.712789 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236b40b0-1a51-4f3a-8615-ac2a76a219aa None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:40.713131 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236b40b0-1a51-4f3a-8615-ac2a76a219aa None admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:40.713899 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236b40b0-1a51-4f3a-8615-ac2a76a219aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:40.714203 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236b40b0-1a51-4f3a-8615-ac2a76a219aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:40.715605 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236b40b0-1a51-4f3a-8615-ac2a76a219aa None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:40.726565 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 198/397] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Oct 31 12:20:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:40.727320 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236b40b0-1a51-4f3a-8615-ac2a76a219aa None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:40.741512 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:40.763797 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:40.784213 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6b073d-14c0-4a7e-841d-d338b496c578 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:40.818344 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd67c8f6-8a72-4d63-896e-ba2a4616e306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:40.818938 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd67c8f6-8a72-4d63-896e-ba2a4616e306 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:40.819244 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd67c8f6-8a72-4d63-896e-ba2a4616e306 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:40.819517 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd67c8f6-8a72-4d63-896e-ba2a4616e306 None admin] PATH_INFO: `/v3/domains` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:40.820150 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd67c8f6-8a72-4d63-896e-ba2a4616e306 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:40.820449 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd67c8f6-8a72-4d63-896e-ba2a4616e306 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:40.821797 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd67c8f6-8a72-4d63-896e-ba2a4616e306 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:40.828806 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236b40b0-1a51-4f3a-8615-ac2a76a219aa None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:40.833006 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 200/398] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Oct 31 12:20:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:40.833790 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd67c8f6-8a72-4d63-896e-ba2a4616e306 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:40.865427 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42c386e-e0b7-4c55-b57c-450238d5b3fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:40.866647 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42c386e-e0b7-4c55-b57c-450238d5b3fc None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:40.866928 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42c386e-e0b7-4c55-b57c-450238d5b3fc None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:40.867228 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42c386e-e0b7-4c55-b57c-450238d5b3fc None admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:40.867951 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42c386e-e0b7-4c55-b57c-450238d5b3fc None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:40.868254 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42c386e-e0b7-4c55-b57c-450238d5b3fc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:40.870358 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42c386e-e0b7-4c55-b57c-450238d5b3fc None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:40.887055 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 199/399] 10.222.0.15 () {56 vars in 1137 bytes} [Mon Oct 31 12:20:40 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:40.888504 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42c386e-e0b7-4c55-b57c-450238d5b3fc None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:40.931515 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd67c8f6-8a72-4d63-896e-ba2a4616e306 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:40.948428 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:40.965988 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c14827-0108-4569-98f4-84a3ea856776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:40.966602 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c14827-0108-4569-98f4-84a3ea856776 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:40.966882 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c14827-0108-4569-98f4-84a3ea856776 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:40.967148 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c14827-0108-4569-98f4-84a3ea856776 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:40.974330 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c14827-0108-4569-98f4-84a3ea856776 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:40.975801 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c14827-0108-4569-98f4-84a3ea856776 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:40.979461 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:40.987091 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-99c14827-0108-4569-98f4-84a3ea856776 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 31 12:20:40.987091 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:40.987091 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:40.987091 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:40.987091 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:40.987091 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:40.987091 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:40.987091 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:40.987091 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:40.987091 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:40.987091 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:40.987091 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:40.987091 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 31 12:20:40.987091 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 31 12:20:40.987091 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 31 12:20:40.987091 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 31 12:20:40.988149 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:40.988149 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:40.988149 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 31 12:20:40.988149 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 31 12:20:40.988149 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 31 12:20:40.988149 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 31 12:20:40.988149 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:40.988149 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:40.988149 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:40.988149 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:40.988149 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:40.988149 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:40.988149 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:40.988149 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:40.988149 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:40.988149 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:40.988149 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:40.988149 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:40.989246 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:40.989246 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:40.989246 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 31 12:20:40.989246 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 31 12:20:40.989246 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 31 12:20:40.989246 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 31 12:20:40.989246 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 31 12:20:40.989246 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 31 12:20:40.989246 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 31 12:20:40.989246 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:20:40.989951 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42c386e-e0b7-4c55-b57c-450238d5b3fc None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:40.992934 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 201/400] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Oct 31 12:20:40 2022] GET /identity/v3/users/neutron => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:20:40.993651 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c14827-0108-4569-98f4-84a3ea856776 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:41.023795 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f9b72f-421a-4835-a9ec-8c32bc8ba8f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:41.024680 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f9b72f-421a-4835-a9ec-8c32bc8ba8f2 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:41.024956 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f9b72f-421a-4835-a9ec-8c32bc8ba8f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:41.025224 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f9b72f-421a-4835-a9ec-8c32bc8ba8f2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:41.032431 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f9b72f-421a-4835-a9ec-8c32bc8ba8f2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:41.033902 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f9b72f-421a-4835-a9ec-8c32bc8ba8f2 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:41.045479 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-36f9b72f-421a-4835-a9ec-8c32bc8ba8f2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 31 12:20:41.045479 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:41.045479 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:41.045479 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:41.045479 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:41.045479 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:41.045479 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:41.045479 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:41.045479 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:41.045479 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:41.045479 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:41.045479 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:41.045479 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 31 12:20:41.045479 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 31 12:20:41.045479 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 31 12:20:41.046526 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 31 12:20:41.046526 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:41.046526 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:41.046526 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:41.046526 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:41.046526 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:41.046526 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:41.046526 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:41.046526 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:41.046526 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:41.046526 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:41.046526 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:41.046526 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:41.046526 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:41.046526 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:41.046526 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:41.046526 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:41.047608 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 31 12:20:41.047608 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 31 12:20:41.047608 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 31 12:20:41.047608 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 31 12:20:41.047608 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 31 12:20:41.047608 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 31 12:20:41.047608 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 31 12:20:41.047608 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:20:41.047608 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 200/401] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Oct 31 12:20:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:20:41.047608 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f9b72f-421a-4835-a9ec-8c32bc8ba8f2 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:41.100223 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c14827-0108-4569-98f4-84a3ea856776 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:41.132745 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d517744f-321f-4247-a7ab-f3fa5ce290b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:41.133338 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d517744f-321f-4247-a7ab-f3fa5ce290b8 None admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:41.133616 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d517744f-321f-4247-a7ab-f3fa5ce290b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:41.133885 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d517744f-321f-4247-a7ab-f3fa5ce290b8 None admin] PATH_INFO: `/v3/users` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:41.134517 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d517744f-321f-4247-a7ab-f3fa5ce290b8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:41.136759 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d517744f-321f-4247-a7ab-f3fa5ce290b8 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:41.146734 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f9b72f-421a-4835-a9ec-8c32bc8ba8f2 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:41.158294 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 202/402] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Oct 31 12:20:40 2022] POST /identity/v3/users => generated 247 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:20:41.159093 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d517744f-321f-4247-a7ab-f3fa5ce290b8 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:41.186684 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b85809-682b-46ba-8a17-bd54d4a9bfd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:41.187383 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b85809-682b-46ba-8a17-bd54d4a9bfd4 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:41.187665 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b85809-682b-46ba-8a17-bd54d4a9bfd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:41.187939 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b85809-682b-46ba-8a17-bd54d4a9bfd4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:41.195718 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b85809-682b-46ba-8a17-bd54d4a9bfd4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:41.197180 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b85809-682b-46ba-8a17-bd54d4a9bfd4 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:41.197449 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:41.214282 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-81b85809-682b-46ba-8a17-bd54d4a9bfd4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 31 12:20:41.214282 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:41.214282 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:41.214282 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:41.214282 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:41.214282 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:41.214282 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:41.214282 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:41.214282 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:41.214282 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:41.214282 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:41.214282 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:41.214282 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 31 12:20:41.214282 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 31 12:20:41.214282 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 31 12:20:41.215436 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 31 12:20:41.215436 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:41.215436 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:41.215436 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:41.215436 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:41.215436 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:41.215436 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:41.215436 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:41.215436 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:41.215436 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:41.215436 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:41.215436 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:41.215436 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:41.215436 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:41.215436 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:41.215436 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:41.215436 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:41.216449 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 31 12:20:41.216449 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 31 12:20:41.216449 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 31 12:20:41.216449 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 31 12:20:41.216449 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 31 12:20:41.216449 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 31 12:20:41.216449 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 31 12:20:41.216449 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:20:41.217241 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 201/403] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Oct 31 12:20:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:20:41.218115 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b85809-682b-46ba-8a17-bd54d4a9bfd4 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:41.237977 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:41.282817 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d517744f-321f-4247-a7ab-f3fa5ce290b8 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:41.319862 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb01b58b-a671-4cbd-a604-8948b9126907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:41.320680 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb01b58b-a671-4cbd-a604-8948b9126907 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:41.321056 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb01b58b-a671-4cbd-a604-8948b9126907 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:41.321412 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb01b58b-a671-4cbd-a604-8948b9126907 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:41.334953 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb01b58b-a671-4cbd-a604-8948b9126907 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:41.336469 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb01b58b-a671-4cbd-a604-8948b9126907 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:41.337546 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b85809-682b-46ba-8a17-bd54d4a9bfd4 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:41.351610 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-fb01b58b-a671-4cbd-a604-8948b9126907 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:20:41.351610 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:41.351610 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:41.351610 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:41.351610 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:41.351610 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:41.351610 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:41.351610 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:41.351610 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:41.351610 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:41.351610 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:41.351610 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:41.351610 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 31 12:20:41.351610 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 31 12:20:41.351610 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 31 12:20:41.351610 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 31 12:20:41.352683 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:41.352683 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:41.352683 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:41.352683 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:41.352683 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:41.352683 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:41.352683 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:41.352683 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:41.352683 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:41.352683 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:41.352683 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:41.352683 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:41.352683 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:41.352683 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:41.352683 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:41.352683 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:41.352683 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 31 12:20:41.353939 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 31 12:20:41.353939 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:20:41.353939 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:20:41.353939 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 203/404] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Oct 31 12:20:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:20:41.353939 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb01b58b-a671-4cbd-a604-8948b9126907 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:41.378996 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f570a98c-107d-4082-8b11-3536ead1475f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:41.379959 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f570a98c-107d-4082-8b11-3536ead1475f None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:41.380235 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f570a98c-107d-4082-8b11-3536ead1475f None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:41.380503 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f570a98c-107d-4082-8b11-3536ead1475f None admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:41.381217 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f570a98c-107d-4082-8b11-3536ead1475f None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:41.381516 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f570a98c-107d-4082-8b11-3536ead1475f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:41.382933 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f570a98c-107d-4082-8b11-3536ead1475f None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:41.401808 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 202/405] 10.222.0.15 () {56 vars in 1139 bytes} [Mon Oct 31 12:20:41 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:41.402908 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f570a98c-107d-4082-8b11-3536ead1475f None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:41.439470 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:41.456388 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb01b58b-a671-4cbd-a604-8948b9126907 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:41.479259 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:41.490596 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4a24f1-9e2e-457a-829d-90e7ab787171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:41.491130 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4a24f1-9e2e-457a-829d-90e7ab787171 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:41.491437 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4a24f1-9e2e-457a-829d-90e7ab787171 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:41.491711 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4a24f1-9e2e-457a-829d-90e7ab787171 None admin] PATH_INFO: `/v3/projects` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:41.492424 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4a24f1-9e2e-457a-829d-90e7ab787171 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:41.492703 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4a24f1-9e2e-457a-829d-90e7ab787171 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:41.494117 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4a24f1-9e2e-457a-829d-90e7ab787171 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:41.512233 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f570a98c-107d-4082-8b11-3536ead1475f None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:41.517245 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 204/406] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Oct 31 12:20:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:41.517245 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4a24f1-9e2e-457a-829d-90e7ab787171 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:41.552715 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20c7ed0-ee79-4710-95eb-069ffa79e74a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:41.553326 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20c7ed0-ee79-4710-95eb-069ffa79e74a None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:41.553599 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20c7ed0-ee79-4710-95eb-069ffa79e74a None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:41.553866 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20c7ed0-ee79-4710-95eb-069ffa79e74a None admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:41.554539 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20c7ed0-ee79-4710-95eb-069ffa79e74a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:41.554835 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20c7ed0-ee79-4710-95eb-069ffa79e74a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:41.556310 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20c7ed0-ee79-4710-95eb-069ffa79e74a None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:41.579662 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 203/407] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Oct 31 12:20:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:41.579662 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20c7ed0-ee79-4710-95eb-069ffa79e74a None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:41.615293 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4a24f1-9e2e-457a-829d-90e7ab787171 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:41.651017 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8dbf06-0e52-4257-8b0e-6e7ca948edd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:41.651671 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8dbf06-0e52-4257-8b0e-6e7ca948edd3 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:41.651944 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8dbf06-0e52-4257-8b0e-6e7ca948edd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:41.652208 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8dbf06-0e52-4257-8b0e-6e7ca948edd3 None admin] PATH_INFO: `/v3/domains` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:41.652847 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8dbf06-0e52-4257-8b0e-6e7ca948edd3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:41.653142 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8dbf06-0e52-4257-8b0e-6e7ca948edd3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:41.654488 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8dbf06-0e52-4257-8b0e-6e7ca948edd3 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:41.665899 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 205/408] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Oct 31 12:20:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:41.667536 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8dbf06-0e52-4257-8b0e-6e7ca948edd3 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:41.677882 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:41.694101 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20c7ed0-ee79-4710-95eb-069ffa79e74a None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:41.723602 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:41.730034 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f3525e-3555-4330-99c9-95785183cc9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:41.730666 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f3525e-3555-4330-99c9-95785183cc9b None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:41.730998 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f3525e-3555-4330-99c9-95785183cc9b None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:41.731297 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f3525e-3555-4330-99c9-95785183cc9b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:41.741728 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f3525e-3555-4330-99c9-95785183cc9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:41.743105 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f3525e-3555-4330-99c9-95785183cc9b None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:41.760036 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-90f3525e-3555-4330-99c9-95785183cc9b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:20:41.760036 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:41.760036 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:41.760036 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:41.760036 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:41.760036 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:41.760036 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:41.760036 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:41.760036 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:41.760036 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:41.760036 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:41.760036 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:41.760036 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 31 12:20:41.760036 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 31 12:20:41.760036 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 31 12:20:41.760036 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 31 12:20:41.761351 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:41.761351 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:41.761351 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:41.761351 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:41.761351 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:41.761351 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:41.761351 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:41.761351 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:41.761351 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:41.761351 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:41.761351 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:41.761351 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:41.761351 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:41.761351 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:41.761351 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:41.761351 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:41.761351 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 31 12:20:41.762399 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 31 12:20:41.762399 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:20:41.762399 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:20:41.765784 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 204/409] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Oct 31 12:20:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:20:41.766586 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f3525e-3555-4330-99c9-95785183cc9b None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:41.783976 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8dbf06-0e52-4257-8b0e-6e7ca948edd3 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:41.819464 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8238e8ea-12cd-4e6b-b128-5797863ea7eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:41.820946 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8238e8ea-12cd-4e6b-b128-5797863ea7eb None admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:41.821240 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8238e8ea-12cd-4e6b-b128-5797863ea7eb None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:41.821514 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8238e8ea-12cd-4e6b-b128-5797863ea7eb None admin] PATH_INFO: `/v3/projects/2c32444cef584bce966b38203ce128b2/users/20d6d7ea077d49e8a66c6c6762499702/roles/9674a2c5acc74dfeb93cfd67dfd890dd` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:41.837592 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8238e8ea-12cd-4e6b-b128-5797863ea7eb None admin] RBAC: Authorizing `identity:create_grant(project_id=2c32444cef584bce966b38203ce128b2, user_id=20d6d7ea077d49e8a66c6c6762499702, role_id=9674a2c5acc74dfeb93cfd67dfd890dd)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:41.839746 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8238e8ea-12cd-4e6b-b128-5797863ea7eb None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:41.863195 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 206/410] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Oct 31 12:20:41 2022] PUT /identity/v3/projects/2c32444cef584bce966b38203ce128b2/users/20d6d7ea077d49e8a66c6c6762499702/roles/9674a2c5acc74dfeb93cfd67dfd890dd => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:20:41.863195 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8238e8ea-12cd-4e6b-b128-5797863ea7eb None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:41.887829 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f3525e-3555-4330-99c9-95785183cc9b None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:41.930634 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06397cac-3264-4279-9e4d-1c3ba6a081bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:41.931472 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06397cac-3264-4279-9e4d-1c3ba6a081bb None admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:41.932494 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06397cac-3264-4279-9e4d-1c3ba6a081bb None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:41.932963 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06397cac-3264-4279-9e4d-1c3ba6a081bb None admin] PATH_INFO: `/v3/projects/2c32444cef584bce966b38203ce128b2/users/99889b1dab0b471d9c77887cffefc652/roles/9674a2c5acc74dfeb93cfd67dfd890dd` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:41.940598 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:41.957132 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06397cac-3264-4279-9e4d-1c3ba6a081bb None admin] RBAC: Authorizing `identity:create_grant(project_id=2c32444cef584bce966b38203ce128b2, user_id=99889b1dab0b471d9c77887cffefc652, role_id=9674a2c5acc74dfeb93cfd67dfd890dd)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:41.958996 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06397cac-3264-4279-9e4d-1c3ba6a081bb None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:41.981367 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 205/411] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Oct 31 12:20:41 2022] PUT /identity/v3/projects/2c32444cef584bce966b38203ce128b2/users/99889b1dab0b471d9c77887cffefc652/roles/9674a2c5acc74dfeb93cfd67dfd890dd => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:20:41.981367 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06397cac-3264-4279-9e4d-1c3ba6a081bb None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:42.038148 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8238e8ea-12cd-4e6b-b128-5797863ea7eb None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:42.058618 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:42.096369 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729ca572-4617-4e18-8495-f03cba17c51a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:42.097325 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729ca572-4617-4e18-8495-f03cba17c51a None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:42.097850 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729ca572-4617-4e18-8495-f03cba17c51a None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:42.098420 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729ca572-4617-4e18-8495-f03cba17c51a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:42.106621 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729ca572-4617-4e18-8495-f03cba17c51a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:42.108165 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729ca572-4617-4e18-8495-f03cba17c51a None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:42.124409 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-729ca572-4617-4e18-8495-f03cba17c51a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 31 12:20:42.124409 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:42.124409 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:42.124409 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:42.124409 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:42.124409 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:42.124409 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:42.124409 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:42.124409 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:42.124409 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:42.124409 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:42.124409 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:42.124409 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 31 12:20:42.124409 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 31 12:20:42.124409 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 31 12:20:42.125341 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 31 12:20:42.125341 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:42.125341 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:42.125341 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:42.125341 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:42.125341 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:42.125341 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:42.125341 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:42.125341 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:42.125341 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:42.125341 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:42.125341 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:42.125341 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:42.125341 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:42.125341 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:42.125341 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:42.125341 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:42.126211 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 31 12:20:42.126211 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 31 12:20:42.126211 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 31 12:20:42.126211 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 31 12:20:42.126211 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 31 12:20:42.126211 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 31 12:20:42.126211 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 31 12:20:42.126211 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:20:42.130282 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 207/412] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Oct 31 12:20:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:20:42.131597 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729ca572-4617-4e18-8495-f03cba17c51a None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:42.148126 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06397cac-3264-4279-9e4d-1c3ba6a081bb None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:42.199189 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1b3d57-8c98-4ebe-b627-814c93e1744b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:42.199189 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1b3d57-8c98-4ebe-b627-814c93e1744b None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:42.199189 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1b3d57-8c98-4ebe-b627-814c93e1744b None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:42.199189 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1b3d57-8c98-4ebe-b627-814c93e1744b None admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:42.199586 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1b3d57-8c98-4ebe-b627-814c93e1744b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:42.199586 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1b3d57-8c98-4ebe-b627-814c93e1744b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:42.199706 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1b3d57-8c98-4ebe-b627-814c93e1744b None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:42.218321 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 206/413] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Oct 31 12:20:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:42.222221 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1b3d57-8c98-4ebe-b627-814c93e1744b None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:42.258570 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729ca572-4617-4e18-8495-f03cba17c51a None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:42.283209 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:42.301965 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b96ca8d-828c-49e9-8fef-d97775593b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:42.303115 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b96ca8d-828c-49e9-8fef-d97775593b54 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:42.304535 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b96ca8d-828c-49e9-8fef-d97775593b54 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:42.305020 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b96ca8d-828c-49e9-8fef-d97775593b54 None admin] PATH_INFO: `/v3/projects` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:42.305905 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b96ca8d-828c-49e9-8fef-d97775593b54 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:42.306385 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b96ca8d-828c-49e9-8fef-d97775593b54 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:42.309143 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b96ca8d-828c-49e9-8fef-d97775593b54 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:42.326788 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 208/414] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Oct 31 12:20:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 31 12:20:42.333369 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:42.334184 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b96ca8d-828c-49e9-8fef-d97775593b54 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:42.355979 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1b3d57-8c98-4ebe-b627-814c93e1744b None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:42.432478 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfe5f8e-4b55-4089-b837-f98e54aa5a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:42.433318 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfe5f8e-4b55-4089-b837-f98e54aa5a92 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:42.433593 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfe5f8e-4b55-4089-b837-f98e54aa5a92 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:42.433855 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfe5f8e-4b55-4089-b837-f98e54aa5a92 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:42.450467 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfe5f8e-4b55-4089-b837-f98e54aa5a92 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:42.453245 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfe5f8e-4b55-4089-b837-f98e54aa5a92 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:42.477838 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-dcfe5f8e-4b55-4089-b837-f98e54aa5a92 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 31 12:20:42.477838 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:42.477838 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:42.477838 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:42.477838 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:42.477838 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:42.477838 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:42.477838 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:42.477838 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:42.477838 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:42.477838 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:42.477838 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:42.477838 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 31 12:20:42.477838 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 31 12:20:42.477838 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 31 12:20:42.479387 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 31 12:20:42.479387 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:42.479387 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:42.479387 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:42.479387 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:42.479387 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:42.479387 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:42.479387 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:42.479387 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:42.479387 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:42.479387 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:42.479387 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:42.479387 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:42.479387 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:42.479387 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:42.479387 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:42.479387 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:42.480406 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 31 12:20:42.480406 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 31 12:20:42.480406 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 31 12:20:42.480406 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 31 12:20:42.480406 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 31 12:20:42.480406 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 31 12:20:42.480406 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 31 12:20:42.480406 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:20:42.489254 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 207/415] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Oct 31 12:20:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:20:42.490631 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfe5f8e-4b55-4089-b837-f98e54aa5a92 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:42.539854 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b96ca8d-828c-49e9-8fef-d97775593b54 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:42.605563 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bb4e9b-e71e-4fc5-b3f2-55a874ad8adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:42.608165 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bb4e9b-e71e-4fc5-b3f2-55a874ad8adc None admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:42.608446 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bb4e9b-e71e-4fc5-b3f2-55a874ad8adc None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:42.608717 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bb4e9b-e71e-4fc5-b3f2-55a874ad8adc None admin] PATH_INFO: `/v3/projects/2c32444cef584bce966b38203ce128b2/users/4feaa01c5ed64b65b10d4dece823702c/roles/9674a2c5acc74dfeb93cfd67dfd890dd` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:42.642652 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bb4e9b-e71e-4fc5-b3f2-55a874ad8adc None admin] RBAC: Authorizing `identity:create_grant(project_id=2c32444cef584bce966b38203ce128b2, user_id=4feaa01c5ed64b65b10d4dece823702c, role_id=9674a2c5acc74dfeb93cfd67dfd890dd)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:42.644364 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bb4e9b-e71e-4fc5-b3f2-55a874ad8adc None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:42.672093 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 209/416] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Oct 31 12:20:42 2022] PUT /identity/v3/projects/2c32444cef584bce966b38203ce128b2/users/4feaa01c5ed64b65b10d4dece823702c/roles/9674a2c5acc74dfeb93cfd67dfd890dd => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 31 12:20:42.673717 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfe5f8e-4b55-4089-b837-f98e54aa5a92 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:42.711997 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b188d822-eb24-4e7a-aebc-2f06f36d74f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:42.712627 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b188d822-eb24-4e7a-aebc-2f06f36d74f3 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:42.713476 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b188d822-eb24-4e7a-aebc-2f06f36d74f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:42.713773 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b188d822-eb24-4e7a-aebc-2f06f36d74f3 None admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:42.714566 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b188d822-eb24-4e7a-aebc-2f06f36d74f3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:42.714876 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b188d822-eb24-4e7a-aebc-2f06f36d74f3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:42.716391 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b188d822-eb24-4e7a-aebc-2f06f36d74f3 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:42.730585 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 208/417] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Oct 31 12:20:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 31 12:20:42.736220 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bb4e9b-e71e-4fc5-b3f2-55a874ad8adc None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:42.767548 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:42.859675 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bb4e9b-e71e-4fc5-b3f2-55a874ad8adc None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:42.898889 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e04796b-c1d5-4778-91e8-9c2735f9929a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:42.899607 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e04796b-c1d5-4778-91e8-9c2735f9929a None admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:42.899887 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e04796b-c1d5-4778-91e8-9c2735f9929a None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:42.900163 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e04796b-c1d5-4778-91e8-9c2735f9929a None admin] PATH_INFO: `/v3/projects/2c32444cef584bce966b38203ce128b2/users/25e9c7d1e4714bf18a1ced84855423d5/roles/9674a2c5acc74dfeb93cfd67dfd890dd` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:42.918671 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e04796b-c1d5-4778-91e8-9c2735f9929a None admin] RBAC: Authorizing `identity:create_grant(project_id=2c32444cef584bce966b38203ce128b2, user_id=25e9c7d1e4714bf18a1ced84855423d5, role_id=9674a2c5acc74dfeb93cfd67dfd890dd)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:42.920193 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e04796b-c1d5-4778-91e8-9c2735f9929a None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:42.942728 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 210/418] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Oct 31 12:20:42 2022] PUT /identity/v3/projects/2c32444cef584bce966b38203ce128b2/users/25e9c7d1e4714bf18a1ced84855423d5/roles/9674a2c5acc74dfeb93cfd67dfd890dd => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:20:43.499402 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdbdf87-0254-4947-82b7-4830037853d1 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:43.499942 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdbdf87-0254-4947-82b7-4830037853d1 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:43.499942 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdbdf87-0254-4947-82b7-4830037853d1 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:43.501410 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 209/419] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:20:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:20:43.506799 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fecfcda-d2d3-4eaa-bbdd-5c88df6785fe None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:43.507092 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fecfcda-d2d3-4eaa-bbdd-5c88df6785fe None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:43.507394 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fecfcda-d2d3-4eaa-bbdd-5c88df6785fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:43.532354 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-5fecfcda-d2d3-4eaa-bbdd-5c88df6785fe None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:20:43.533543 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-5fecfcda-d2d3-4eaa-bbdd-5c88df6785fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:20:43.629168 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 211/420] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:20:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 31 12:20:43.980839 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdbdf87-0254-4947-82b7-4830037853d1 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:44.000818 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:44.120618 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdbdf87-0254-4947-82b7-4830037853d1 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:44.169227 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8d168d-e5ed-4509-8c64-155c11eeb70d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:44.170129 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8d168d-e5ed-4509-8c64-155c11eeb70d None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:44.170547 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8d168d-e5ed-4509-8c64-155c11eeb70d None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:44.170970 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8d168d-e5ed-4509-8c64-155c11eeb70d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:44.189057 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8d168d-e5ed-4509-8c64-155c11eeb70d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 31 12:20:44.194072 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-fa8d168d-e5ed-4509-8c64-155c11eeb70d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 31 12:20:44.194072 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:44.194072 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:44.194072 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:44.194072 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:44.194072 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:44.194072 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:44.194072 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:44.194072 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:44.194072 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:44.194072 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:44.194072 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:44.194072 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 31 12:20:44.194072 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 31 12:20:44.194072 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 31 12:20:44.194072 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 31 12:20:44.195125 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 31 12:20:44.195125 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 31 12:20:44.195125 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 31 12:20:44.195125 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 31 12:20:44.195125 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:44.195125 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:44.195125 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:44.195125 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:44.195125 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:44.195125 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:44.195125 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:44.195125 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:44.195125 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:44.195125 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:44.195125 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:44.195125 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:44.196127 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:44.196127 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:44.196127 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:44.196127 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:44.196127 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 31 12:20:44.196127 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 31 12:20:44.196127 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 31 12:20:44.196127 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 31 12:20:44.196127 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 31 12:20:44.196127 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 31 12:20:44.196127 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 31 12:20:44.196127 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:20:44.198156 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 210/421] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Oct 31 12:20:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:20:44.204148 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fecfcda-d2d3-4eaa-bbdd-5c88df6785fe None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:44.214042 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:44.343144 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fecfcda-d2d3-4eaa-bbdd-5c88df6785fe None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:44.402099 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da6fd8e-3057-4a5a-bc34-8dd418bd1ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:44.402707 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da6fd8e-3057-4a5a-bc34-8dd418bd1ff6 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:44.402986 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da6fd8e-3057-4a5a-bc34-8dd418bd1ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:44.403286 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da6fd8e-3057-4a5a-bc34-8dd418bd1ff6 None admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:44.403951 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da6fd8e-3057-4a5a-bc34-8dd418bd1ff6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:44.404250 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da6fd8e-3057-4a5a-bc34-8dd418bd1ff6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:44.405608 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da6fd8e-3057-4a5a-bc34-8dd418bd1ff6 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:44.419689 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 212/422] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Oct 31 12:20:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:44.425728 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8d168d-e5ed-4509-8c64-155c11eeb70d None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:44.509961 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:44.552113 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8d168d-e5ed-4509-8c64-155c11eeb70d None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:44.608627 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a72c80f-c2e6-4a82-ba28-1fb9aa331e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:44.609413 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a72c80f-c2e6-4a82-ba28-1fb9aa331e77 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:44.609693 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a72c80f-c2e6-4a82-ba28-1fb9aa331e77 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:44.609966 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a72c80f-c2e6-4a82-ba28-1fb9aa331e77 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:44.624703 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a72c80f-c2e6-4a82-ba28-1fb9aa331e77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:44.627200 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a72c80f-c2e6-4a82-ba28-1fb9aa331e77 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:44.647493 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-8a72c80f-c2e6-4a82-ba28-1fb9aa331e77 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:20:44.647493 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:44.647493 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:44.647493 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:44.647493 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:44.647493 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:44.647493 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:44.647493 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:44.647493 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:44.647493 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:44.647493 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:44.647493 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:44.647493 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 31 12:20:44.647493 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 31 12:20:44.647493 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 31 12:20:44.647493 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 31 12:20:44.648485 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:44.648485 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:44.648485 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:44.648485 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:44.648485 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:44.648485 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:44.648485 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:44.648485 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:44.648485 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:44.648485 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:44.648485 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:44.648485 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:44.648485 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:44.648485 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:44.648485 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:44.648485 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:44.648485 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 31 12:20:44.650860 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 31 12:20:44.650860 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:20:44.650860 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:20:44.650860 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 211/423] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Oct 31 12:20:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:20:44.654614 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da6fd8e-3057-4a5a-bc34-8dd418bd1ff6 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:44.752877 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:44.771885 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e91923-e5b1-422a-881b-237fd4f482e8 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:44.772162 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e91923-e5b1-422a-881b-237fd4f482e8 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:44.772538 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e91923-e5b1-422a-881b-237fd4f482e8 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:44.774092 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 212/424] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:20:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 31 12:20:44.779720 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943a9b5d-ef25-4c53-a049-8abcaebc0fa4 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:44.779991 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943a9b5d-ef25-4c53-a049-8abcaebc0fa4 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:44.780267 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943a9b5d-ef25-4c53-a049-8abcaebc0fa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:44.800983 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-943a9b5d-ef25-4c53-a049-8abcaebc0fa4 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:20:44.802218 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-943a9b5d-ef25-4c53-a049-8abcaebc0fa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:20:44.808567 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da6fd8e-3057-4a5a-bc34-8dd418bd1ff6 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:44.888097 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fab061c-3c32-454e-addf-4e7682217dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:44.889733 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fab061c-3c32-454e-addf-4e7682217dcd None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:44.890338 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fab061c-3c32-454e-addf-4e7682217dcd None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:44.890799 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fab061c-3c32-454e-addf-4e7682217dcd None admin] PATH_INFO: `/v3/domains` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:44.896195 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fab061c-3c32-454e-addf-4e7682217dcd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:44.896741 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fab061c-3c32-454e-addf-4e7682217dcd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:44.898977 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fab061c-3c32-454e-addf-4e7682217dcd None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:44.920927 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 213/425] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Oct 31 12:20:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:44.930393 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fab061c-3c32-454e-addf-4e7682217dcd None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:44.959924 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 213/426] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:20:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 31 12:20:44.961408 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be9f113-4a87-4f46-85f5-a8e4e1aebce3 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:44.961688 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be9f113-4a87-4f46-85f5-a8e4e1aebce3 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:44.961953 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be9f113-4a87-4f46-85f5-a8e4e1aebce3 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:44.963259 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 214/427] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:20:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:20:44.969714 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca91362d-67e6-4b1f-a4d9-9cd1ef76b591 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:44.969999 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca91362d-67e6-4b1f-a4d9-9cd1ef76b591 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:44.970275 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca91362d-67e6-4b1f-a4d9-9cd1ef76b591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:45.003808 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-ca91362d-67e6-4b1f-a4d9-9cd1ef76b591 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:20:45.005855 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-ca91362d-67e6-4b1f-a4d9-9cd1ef76b591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:20:45.127695 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:45.133237 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fab061c-3c32-454e-addf-4e7682217dcd None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:45.160721 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 215/428] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:20:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 31 12:20:45.220440 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:45.241373 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9151a53a-d57e-4fb4-ab61-a2381d3c1090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:45.242478 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9151a53a-d57e-4fb4-ab61-a2381d3c1090 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:45.242937 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9151a53a-d57e-4fb4-ab61-a2381d3c1090 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:45.243433 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9151a53a-d57e-4fb4-ab61-a2381d3c1090 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:45.258198 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9151a53a-d57e-4fb4-ab61-a2381d3c1090 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:45.258198 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9151a53a-d57e-4fb4-ab61-a2381d3c1090 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:45.285488 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-9151a53a-d57e-4fb4-ab61-a2381d3c1090 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 31 12:20:45.285488 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:45.285488 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:45.285488 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:45.285488 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:45.285488 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:45.285488 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:45.285488 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:45.285488 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:45.285488 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:45.285488 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:45.285488 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:45.285488 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 31 12:20:45.285488 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 31 12:20:45.285488 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 31 12:20:45.285488 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 31 12:20:45.286447 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:45.286447 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:45.286447 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 31 12:20:45.286447 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 31 12:20:45.286447 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 31 12:20:45.286447 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 31 12:20:45.286447 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:45.286447 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:45.286447 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:45.286447 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:45.286447 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:45.286447 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:45.286447 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:45.286447 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:45.286447 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:45.286447 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:45.286447 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:45.286447 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:45.287384 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:45.287384 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:45.287384 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 31 12:20:45.287384 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 31 12:20:45.287384 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 31 12:20:45.287384 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 31 12:20:45.287384 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 31 12:20:45.287384 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 31 12:20:45.287384 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 31 12:20:45.287384 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:20:45.289588 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 214/429] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Oct 31 12:20:44 2022] GET /identity/v3/users/swift => generated 83 bytes in 363 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:20:45.300350 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca91362d-67e6-4b1f-a4d9-9cd1ef76b591 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:45.337603 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9151a53a-d57e-4fb4-ab61-a2381d3c1090 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:45.439288 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca91362d-67e6-4b1f-a4d9-9cd1ef76b591 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:45.470162 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb702ef-191c-463b-ae2c-0cb472cd4888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:45.470819 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb702ef-191c-463b-ae2c-0cb472cd4888 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:45.471106 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb702ef-191c-463b-ae2c-0cb472cd4888 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:45.471412 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb702ef-191c-463b-ae2c-0cb472cd4888 None admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:45.472132 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb702ef-191c-463b-ae2c-0cb472cd4888 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:45.472449 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb702ef-191c-463b-ae2c-0cb472cd4888 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:45.473882 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb702ef-191c-463b-ae2c-0cb472cd4888 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:45.493778 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 216/430] 10.222.0.15 () {56 vars in 1135 bytes} [Mon Oct 31 12:20:45 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:45.499250 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9a18fe-b79b-4006-8a7c-00bbff62edb7 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:45.499539 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9a18fe-b79b-4006-8a7c-00bbff62edb7 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:45.500664 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9a18fe-b79b-4006-8a7c-00bbff62edb7 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:45.502179 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 217/431] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:20:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:20:45.502888 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9a18fe-b79b-4006-8a7c-00bbff62edb7 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:45.534500 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9151a53a-d57e-4fb4-ab61-a2381d3c1090 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:45.558625 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:45.571393 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b0f023-5f57-482e-8fe2-c8a9daab6f56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:45.572048 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b0f023-5f57-482e-8fe2-c8a9daab6f56 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:45.572386 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b0f023-5f57-482e-8fe2-c8a9daab6f56 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:45.572744 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b0f023-5f57-482e-8fe2-c8a9daab6f56 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:45.583230 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b0f023-5f57-482e-8fe2-c8a9daab6f56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 31 12:20:45.589002 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-40b0f023-5f57-482e-8fe2-c8a9daab6f56 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 31 12:20:45.589002 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:45.589002 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:45.589002 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:45.589002 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:45.589002 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:45.589002 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:45.589002 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:45.589002 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:45.589002 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:45.589002 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:45.589002 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:45.589002 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 31 12:20:45.589002 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 31 12:20:45.589002 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 31 12:20:45.589002 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 31 12:20:45.589960 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 31 12:20:45.589960 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 31 12:20:45.589960 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 31 12:20:45.589960 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 31 12:20:45.589960 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:45.589960 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:45.589960 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:45.589960 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:45.589960 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:45.589960 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:45.589960 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:45.589960 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:45.589960 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:45.589960 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:45.589960 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:45.589960 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:45.590744 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:45.590744 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:45.590744 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:45.590744 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:45.590744 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 31 12:20:45.590744 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 31 12:20:45.590744 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 31 12:20:45.590744 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 31 12:20:45.590744 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 31 12:20:45.590744 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 31 12:20:45.590744 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 31 12:20:45.590744 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:20:45.594936 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 215/432] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Oct 31 12:20:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:20:45.596225 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578ad689-b81b-44bc-9ab5-21c9bfe46531 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:45.596509 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578ad689-b81b-44bc-9ab5-21c9bfe46531 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:45.596781 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578ad689-b81b-44bc-9ab5-21c9bfe46531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:45.615053 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-578ad689-b81b-44bc-9ab5-21c9bfe46531 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:20:45.616266 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-578ad689-b81b-44bc-9ab5-21c9bfe46531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:20:45.622289 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9a18fe-b79b-4006-8a7c-00bbff62edb7 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:45.659748 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05464ff6-bed6-4e75-94f0-11f34f038f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:45.660803 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05464ff6-bed6-4e75-94f0-11f34f038f15 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:45.661088 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05464ff6-bed6-4e75-94f0-11f34f038f15 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:45.661361 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05464ff6-bed6-4e75-94f0-11f34f038f15 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:45.664128 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:45.674048 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05464ff6-bed6-4e75-94f0-11f34f038f15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:45.675957 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05464ff6-bed6-4e75-94f0-11f34f038f15 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:45.692397 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-05464ff6-bed6-4e75-94f0-11f34f038f15 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:20:45.692397 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:45.692397 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:45.692397 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:45.692397 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:45.692397 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:45.692397 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:45.692397 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:45.692397 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:45.692397 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:45.692397 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:45.692397 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:45.692397 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 31 12:20:45.692397 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 31 12:20:45.692397 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 31 12:20:45.692397 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 31 12:20:45.693360 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:45.693360 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:45.693360 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:45.693360 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:45.693360 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:45.693360 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:45.693360 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:45.693360 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:45.693360 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:45.693360 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:45.693360 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:45.693360 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:45.693360 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:45.693360 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:45.693360 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:45.693360 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:45.693360 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 31 12:20:45.699402 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 31 12:20:45.699402 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:20:45.699402 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:20:45.699402 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 218/433] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Oct 31 12:20:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:20:45.699402 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05464ff6-bed6-4e75-94f0-11f34f038f15 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:45.699671 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 216/434] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:20:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 31 12:20:45.700572 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578ad689-b81b-44bc-9ab5-21c9bfe46531 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:45.822312 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:45.831095 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05464ff6-bed6-4e75-94f0-11f34f038f15 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:45.832835 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578ad689-b81b-44bc-9ab5-21c9bfe46531 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:45.867788 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138f12a2-e8cc-4be6-87d4-d57481a56d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:45.868448 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138f12a2-e8cc-4be6-87d4-d57481a56d3a None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:45.868689 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138f12a2-e8cc-4be6-87d4-d57481a56d3a None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:45.868960 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138f12a2-e8cc-4be6-87d4-d57481a56d3a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:45.869556 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1a8592-4e8e-4d7f-8ad6-e84e62ce7f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:45.870148 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1a8592-4e8e-4d7f-8ad6-e84e62ce7f76 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:45.870420 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1a8592-4e8e-4d7f-8ad6-e84e62ce7f76 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:45.870689 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1a8592-4e8e-4d7f-8ad6-e84e62ce7f76 None admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:45.871490 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1a8592-4e8e-4d7f-8ad6-e84e62ce7f76 None admin] RBAC: Adding query filter params (name=service) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:45.871900 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1a8592-4e8e-4d7f-8ad6-e84e62ce7f76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:45.873240 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1a8592-4e8e-4d7f-8ad6-e84e62ce7f76 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:45.879346 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138f12a2-e8cc-4be6-87d4-d57481a56d3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 31 12:20:45.887122 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 217/435] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Oct 31 12:20:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:45.887122 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1a8592-4e8e-4d7f-8ad6-e84e62ce7f76 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:45.887356 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-138f12a2-e8cc-4be6-87d4-d57481a56d3a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 31 12:20:45.887356 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:45.887356 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:45.887356 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:45.887356 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:45.887356 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:45.887356 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:45.887356 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:45.887356 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:45.887356 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:45.887356 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:45.887356 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:45.887356 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 31 12:20:45.887356 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 31 12:20:45.887356 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 31 12:20:45.887356 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 31 12:20:45.888165 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 31 12:20:45.888165 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 31 12:20:45.888165 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 31 12:20:45.888165 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 31 12:20:45.888165 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:45.888165 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:45.888165 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:45.888165 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:45.888165 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:45.888165 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:45.888165 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:45.888165 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:45.888165 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:45.888165 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:45.888165 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:45.888165 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:45.888940 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:45.888940 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:45.888940 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:45.888940 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:45.888940 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 31 12:20:45.888940 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 31 12:20:45.888940 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 31 12:20:45.888940 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 31 12:20:45.888940 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 31 12:20:45.888940 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 31 12:20:45.888940 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 31 12:20:45.888940 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:20:45.888940 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 219/436] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Oct 31 12:20:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:20:45.888940 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcb9986-f422-44ef-a3c8-301dacd3d765 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:45.889647 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcb9986-f422-44ef-a3c8-301dacd3d765 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:45.889647 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcb9986-f422-44ef-a3c8-301dacd3d765 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:45.890300 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 220/437] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:20:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:20:45.891265 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcb9986-f422-44ef-a3c8-301dacd3d765 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:45.936112 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:45.996267 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcb9986-f422-44ef-a3c8-301dacd3d765 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:45.996720 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1a8592-4e8e-4d7f-8ad6-e84e62ce7f76 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:46.031789 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66db6af-d385-4453-967a-18616a616827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:46.031973 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bad33db-61e2-4aaf-bc94-eebb21309ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:46.032392 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66db6af-d385-4453-967a-18616a616827 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:46.032485 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bad33db-61e2-4aaf-bc94-eebb21309ade None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:46.032666 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66db6af-d385-4453-967a-18616a616827 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:46.032753 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bad33db-61e2-4aaf-bc94-eebb21309ade None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:46.032934 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66db6af-d385-4453-967a-18616a616827 None admin] PATH_INFO: `/v3/domains` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:46.033021 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bad33db-61e2-4aaf-bc94-eebb21309ade None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:46.033565 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66db6af-d385-4453-967a-18616a616827 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:46.033863 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66db6af-d385-4453-967a-18616a616827 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:46.035222 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66db6af-d385-4453-967a-18616a616827 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:46.043951 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bad33db-61e2-4aaf-bc94-eebb21309ade None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:46.045373 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bad33db-61e2-4aaf-bc94-eebb21309ade None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:46.047597 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 218/438] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Oct 31 12:20:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:46.048408 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66db6af-d385-4453-967a-18616a616827 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:46.064846 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-6bad33db-61e2-4aaf-bc94-eebb21309ade None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:20:46.064846 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:46.064846 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:46.064846 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:46.064846 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:46.064846 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:46.064846 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:46.064846 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:46.064846 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:46.064846 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:46.064846 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:46.064846 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:46.064846 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 31 12:20:46.064846 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 31 12:20:46.064846 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 31 12:20:46.064846 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 31 12:20:46.065763 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:46.065763 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:46.065763 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:46.065763 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:46.065763 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:46.065763 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:46.065763 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:46.065763 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:46.065763 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:46.065763 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:46.065763 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:46.065763 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:46.065763 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:46.065763 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:46.065763 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:46.065763 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:46.065763 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 31 12:20:46.066974 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 31 12:20:46.066974 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:20:46.066974 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:20:46.066974 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 221/439] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Oct 31 12:20:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:20:46.067337 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e915984-10a1-493f-a563-1e2691d01d08 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:46.067684 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e915984-10a1-493f-a563-1e2691d01d08 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:46.067780 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e915984-10a1-493f-a563-1e2691d01d08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:46.081281 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:46.086624 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-7e915984-10a1-493f-a563-1e2691d01d08 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:20:46.087840 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-7e915984-10a1-493f-a563-1e2691d01d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:20:46.145332 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:46.148538 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66db6af-d385-4453-967a-18616a616827 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:46.153684 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 222/440] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:20:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 31 12:20:46.154522 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e915984-10a1-493f-a563-1e2691d01d08 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:46.182886 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53a26a9-fba9-42f1-aea7-8c740a6d4673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:46.183519 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53a26a9-fba9-42f1-aea7-8c740a6d4673 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:46.183793 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53a26a9-fba9-42f1-aea7-8c740a6d4673 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:46.184066 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53a26a9-fba9-42f1-aea7-8c740a6d4673 None admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:46.184718 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53a26a9-fba9-42f1-aea7-8c740a6d4673 None admin] RBAC: Adding query filter params (name=service) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:46.185014 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53a26a9-fba9-42f1-aea7-8c740a6d4673 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:46.186361 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53a26a9-fba9-42f1-aea7-8c740a6d4673 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:46.199094 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 219/441] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Oct 31 12:20:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:46.199094 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53a26a9-fba9-42f1-aea7-8c740a6d4673 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:46.281588 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e915984-10a1-493f-a563-1e2691d01d08 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:46.297226 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53a26a9-fba9-42f1-aea7-8c740a6d4673 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:46.317872 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb78f48-2b2f-4658-9f96-b2900ff92392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:46.318508 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb78f48-2b2f-4658-9f96-b2900ff92392 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:46.318782 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb78f48-2b2f-4658-9f96-b2900ff92392 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:46.319049 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb78f48-2b2f-4658-9f96-b2900ff92392 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:46.321783 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:46.331027 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb78f48-2b2f-4658-9f96-b2900ff92392 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 31 12:20:46.333586 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f3f6e7-1c3d-4b15-9246-e90599d43703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:46.334203 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f3f6e7-1c3d-4b15-9246-e90599d43703 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:46.334479 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f3f6e7-1c3d-4b15-9246-e90599d43703 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:46.334744 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f3f6e7-1c3d-4b15-9246-e90599d43703 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:46.335809 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-adb78f48-2b2f-4658-9f96-b2900ff92392 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 31 12:20:46.335809 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:46.335809 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:46.335809 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:46.335809 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:46.335809 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:46.335809 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:46.335809 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:46.335809 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:46.335809 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:46.335809 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:46.335809 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:46.335809 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 31 12:20:46.335809 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 31 12:20:46.335809 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 31 12:20:46.335809 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 31 12:20:46.336686 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 31 12:20:46.336686 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 31 12:20:46.336686 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 31 12:20:46.336686 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 31 12:20:46.336686 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:46.336686 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:46.336686 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:46.336686 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:46.336686 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:46.336686 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:46.336686 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:46.336686 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:46.336686 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:46.336686 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:46.336686 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:46.336686 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:46.337462 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:46.337462 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:46.337462 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:46.337462 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:46.337462 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 31 12:20:46.337462 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 31 12:20:46.337462 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 31 12:20:46.337462 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 31 12:20:46.337462 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 31 12:20:46.337462 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 31 12:20:46.337462 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 31 12:20:46.337462 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:20:46.337462 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 223/442] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Oct 31 12:20:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:20:46.338617 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb78f48-2b2f-4658-9f96-b2900ff92392 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:46.346681 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f3f6e7-1c3d-4b15-9246-e90599d43703 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:46.348154 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f3f6e7-1c3d-4b15-9246-e90599d43703 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:46.359452 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-a6f3f6e7-1c3d-4b15-9246-e90599d43703 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 31 12:20:46.359452 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:46.359452 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:46.359452 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:46.359452 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:46.359452 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:46.359452 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:46.359452 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:46.359452 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:46.359452 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:46.359452 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:46.359452 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:46.359452 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 31 12:20:46.359452 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 31 12:20:46.359452 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 31 12:20:46.360334 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 31 12:20:46.360334 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:46.360334 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:46.360334 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:46.360334 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:46.360334 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:46.360334 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:46.360334 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:46.360334 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:46.360334 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:46.360334 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:46.360334 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:46.360334 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:46.360334 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:46.360334 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:46.360334 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:46.360334 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:46.361177 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 31 12:20:46.361177 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 31 12:20:46.361177 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 31 12:20:46.361177 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 31 12:20:46.361177 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 31 12:20:46.361177 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 31 12:20:46.361177 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 31 12:20:46.361177 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:20:46.361177 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 220/443] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Oct 31 12:20:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:20:46.361646 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f3f6e7-1c3d-4b15-9246-e90599d43703 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:46.391006 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:46.455574 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb78f48-2b2f-4658-9f96-b2900ff92392 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:46.473004 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f3f6e7-1c3d-4b15-9246-e90599d43703 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:46.489767 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262f5081-aad0-4597-9824-3ed1d8ce44bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:46.490367 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262f5081-aad0-4597-9824-3ed1d8ce44bd None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:46.490645 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262f5081-aad0-4597-9824-3ed1d8ce44bd None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:46.490910 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262f5081-aad0-4597-9824-3ed1d8ce44bd None admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:46.491570 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262f5081-aad0-4597-9824-3ed1d8ce44bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:46.491869 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262f5081-aad0-4597-9824-3ed1d8ce44bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:46.493291 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262f5081-aad0-4597-9824-3ed1d8ce44bd None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:46.504573 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 224/444] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Oct 31 12:20:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:46.505363 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262f5081-aad0-4597-9824-3ed1d8ce44bd None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:46.506135 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0b26c6-7001-45a7-8a41-9c22123ed3e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:46.506762 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0b26c6-7001-45a7-8a41-9c22123ed3e8 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:46.507046 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0b26c6-7001-45a7-8a41-9c22123ed3e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:46.507350 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0b26c6-7001-45a7-8a41-9c22123ed3e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:46.519851 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0b26c6-7001-45a7-8a41-9c22123ed3e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:46.521298 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0b26c6-7001-45a7-8a41-9c22123ed3e8 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:46.535573 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-5b0b26c6-7001-45a7-8a41-9c22123ed3e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:20:46.535573 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:46.535573 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:46.535573 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:46.535573 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:46.535573 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:46.535573 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:46.535573 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:46.535573 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:46.535573 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:46.535573 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:46.535573 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:46.535573 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 31 12:20:46.535573 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 31 12:20:46.535573 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 31 12:20:46.535573 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 31 12:20:46.536623 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:46.536623 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:46.536623 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:46.536623 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:46.536623 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:46.536623 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:46.536623 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:46.536623 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:46.536623 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:46.536623 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:46.536623 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:46.536623 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:46.536623 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:46.536623 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:46.536623 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:46.536623 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:46.536623 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 31 12:20:46.537663 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 31 12:20:46.537663 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:20:46.537663 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:20:46.537663 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 221/445] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Oct 31 12:20:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:20:46.537663 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0b26c6-7001-45a7-8a41-9c22123ed3e8 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:46.567386 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:46.610689 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:46.613382 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262f5081-aad0-4597-9824-3ed1d8ce44bd None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:46.635976 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0b26c6-7001-45a7-8a41-9c22123ed3e8 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:46.646295 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd52dec9-fee3-4e5d-8887-5e15e35a59e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:46.646892 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd52dec9-fee3-4e5d-8887-5e15e35a59e2 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:46.647184 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd52dec9-fee3-4e5d-8887-5e15e35a59e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:46.647460 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd52dec9-fee3-4e5d-8887-5e15e35a59e2 None admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:46.648104 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd52dec9-fee3-4e5d-8887-5e15e35a59e2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:46.648400 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd52dec9-fee3-4e5d-8887-5e15e35a59e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:46.649740 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd52dec9-fee3-4e5d-8887-5e15e35a59e2 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:46.658241 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 225/446] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Oct 31 12:20:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:46.659019 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd52dec9-fee3-4e5d-8887-5e15e35a59e2 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:46.674978 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349eb5d7-f639-4acc-bce2-86f348e24cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:46.675721 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349eb5d7-f639-4acc-bce2-86f348e24cd7 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:46.676109 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349eb5d7-f639-4acc-bce2-86f348e24cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:46.676508 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349eb5d7-f639-4acc-bce2-86f348e24cd7 None admin] PATH_INFO: `/v3/projects` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:46.677194 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349eb5d7-f639-4acc-bce2-86f348e24cd7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:46.677498 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349eb5d7-f639-4acc-bce2-86f348e24cd7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:46.678914 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349eb5d7-f639-4acc-bce2-86f348e24cd7 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:46.691569 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 222/447] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Oct 31 12:20:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:46.692359 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349eb5d7-f639-4acc-bce2-86f348e24cd7 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:46.765269 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd52dec9-fee3-4e5d-8887-5e15e35a59e2 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:46.782794 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:46.788859 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349eb5d7-f639-4acc-bce2-86f348e24cd7 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:46.800857 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbea831e-819c-48bf-b959-e234657707cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:46.801485 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbea831e-819c-48bf-b959-e234657707cd None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:46.801762 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbea831e-819c-48bf-b959-e234657707cd None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:46.802026 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbea831e-819c-48bf-b959-e234657707cd None admin] PATH_INFO: `/v3/roles/service` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:46.811842 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbea831e-819c-48bf-b959-e234657707cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 31 12:20:46.817002 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-fbea831e-819c-48bf-b959-e234657707cd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 31 12:20:46.817002 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:46.817002 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:46.817002 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:46.817002 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:46.817002 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:46.817002 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:46.817002 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:46.817002 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:46.817002 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:46.817002 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:46.817002 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:46.817002 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 31 12:20:46.817002 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 31 12:20:46.817002 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 31 12:20:46.817002 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 31 12:20:46.817893 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 31 12:20:46.817893 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 31 12:20:46.817893 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 31 12:20:46.817893 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 31 12:20:46.817893 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:46.817893 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:46.817893 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:46.817893 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:46.817893 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:46.817893 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:46.817893 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:46.817893 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:46.817893 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:46.817893 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:46.817893 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:46.817893 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:46.818687 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:46.818687 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:46.818687 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:46.818687 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:46.818687 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 31 12:20:46.818687 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 31 12:20:46.818687 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 31 12:20:46.818687 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 31 12:20:46.818687 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 31 12:20:46.818687 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 31 12:20:46.818687 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 31 12:20:46.818687 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:20:46.818687 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 226/448] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Oct 31 12:20:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:20:46.819383 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbea831e-819c-48bf-b959-e234657707cd None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:46.824037 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3f7b18-060d-41b0-9ad8-6d636624583c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:46.824666 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3f7b18-060d-41b0-9ad8-6d636624583c None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:46.824944 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3f7b18-060d-41b0-9ad8-6d636624583c None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:46.825212 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3f7b18-060d-41b0-9ad8-6d636624583c None admin] PATH_INFO: `/v3/users/nova` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:46.827305 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:46.833250 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3f7b18-060d-41b0-9ad8-6d636624583c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:46.835047 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3f7b18-060d-41b0-9ad8-6d636624583c None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:46.846821 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-8d3f7b18-060d-41b0-9ad8-6d636624583c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 31 12:20:46.846821 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:46.846821 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:46.846821 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:46.846821 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:46.846821 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:46.846821 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:46.846821 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:46.846821 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:46.846821 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:46.846821 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:46.846821 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:46.846821 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 31 12:20:46.846821 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 31 12:20:46.846821 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 31 12:20:46.846821 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 31 12:20:46.847788 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:46.847788 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:46.847788 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 31 12:20:46.847788 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 31 12:20:46.847788 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 31 12:20:46.847788 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 31 12:20:46.847788 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:46.847788 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:46.847788 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:46.847788 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:46.847788 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:46.847788 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:46.847788 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:46.847788 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:46.847788 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:46.847788 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:46.847788 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:46.847788 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:46.848739 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:46.848739 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:46.848739 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 31 12:20:46.848739 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 31 12:20:46.848739 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 31 12:20:46.848739 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 31 12:20:46.848739 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 31 12:20:46.848739 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 31 12:20:46.848739 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 31 12:20:46.848739 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:20:46.848739 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 223/449] 10.222.0.15 () {56 vars in 1093 bytes} [Mon Oct 31 12:20:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:20:46.848739 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3f7b18-060d-41b0-9ad8-6d636624583c None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:46.919531 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbea831e-819c-48bf-b959-e234657707cd None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:46.945641 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3f7b18-060d-41b0-9ad8-6d636624583c None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:46.953476 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae4a325-7700-40f7-923e-f7649a3437b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:46.954078 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae4a325-7700-40f7-923e-f7649a3437b1 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:46.954358 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae4a325-7700-40f7-923e-f7649a3437b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:46.954638 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae4a325-7700-40f7-923e-f7649a3437b1 None admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:46.955303 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae4a325-7700-40f7-923e-f7649a3437b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:46.955614 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae4a325-7700-40f7-923e-f7649a3437b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:46.957019 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae4a325-7700-40f7-923e-f7649a3437b1 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:46.969450 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 227/450] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Oct 31 12:20:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:46.970215 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae4a325-7700-40f7-923e-f7649a3437b1 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:46.980251 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d04b24-76ae-41e4-9cf7-c7efd6258b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:46.980870 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d04b24-76ae-41e4-9cf7-c7efd6258b13 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:46.981148 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d04b24-76ae-41e4-9cf7-c7efd6258b13 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:46.981419 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d04b24-76ae-41e4-9cf7-c7efd6258b13 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:46.991853 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d04b24-76ae-41e4-9cf7-c7efd6258b13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:46.993293 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d04b24-76ae-41e4-9cf7-c7efd6258b13 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:47.007922 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-c4d04b24-76ae-41e4-9cf7-c7efd6258b13 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:20:47.007922 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:47.007922 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:47.007922 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:47.007922 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:47.007922 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:47.007922 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:47.007922 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:47.007922 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:47.007922 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:47.007922 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:47.007922 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:47.007922 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 31 12:20:47.007922 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 31 12:20:47.007922 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 31 12:20:47.007922 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 31 12:20:47.008986 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:47.008986 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:47.008986 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:47.008986 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:47.008986 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:47.008986 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:47.008986 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:47.008986 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:47.008986 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:47.008986 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:47.008986 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:47.008986 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:47.008986 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:47.008986 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:47.008986 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:47.008986 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:47.008986 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 31 12:20:47.009877 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 31 12:20:47.009877 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:20:47.009877 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:20:47.009877 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 224/451] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Oct 31 12:20:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:20:47.010820 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d04b24-76ae-41e4-9cf7-c7efd6258b13 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:47.011653 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:47.060513 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:47.077471 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae4a325-7700-40f7-923e-f7649a3437b1 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:47.112594 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592ac4ca-f6dd-4c13-8db0-60136577c014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:47.113253 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592ac4ca-f6dd-4c13-8db0-60136577c014 None admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:47.113536 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592ac4ca-f6dd-4c13-8db0-60136577c014 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:47.113649 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d04b24-76ae-41e4-9cf7-c7efd6258b13 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:47.113810 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592ac4ca-f6dd-4c13-8db0-60136577c014 None admin] PATH_INFO: `/v3/projects/2c32444cef584bce966b38203ce128b2/users/b2254a5786c249b4aa79244f7198ae65/roles/9674a2c5acc74dfeb93cfd67dfd890dd` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:47.128355 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592ac4ca-f6dd-4c13-8db0-60136577c014 None admin] RBAC: Authorizing `identity:create_grant(project_id=2c32444cef584bce966b38203ce128b2, user_id=b2254a5786c249b4aa79244f7198ae65, role_id=9674a2c5acc74dfeb93cfd67dfd890dd)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:47.129781 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592ac4ca-f6dd-4c13-8db0-60136577c014 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:47.141956 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 228/452] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Oct 31 12:20:46 2022] PUT /identity/v3/projects/2c32444cef584bce966b38203ce128b2/users/b2254a5786c249b4aa79244f7198ae65/roles/9674a2c5acc74dfeb93cfd67dfd890dd => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:20:47.142754 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592ac4ca-f6dd-4c13-8db0-60136577c014 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:47.150347 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d8bc7e-1a72-483f-bff7-500d6e9de61a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:47.151914 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d8bc7e-1a72-483f-bff7-500d6e9de61a None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:47.152881 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d8bc7e-1a72-483f-bff7-500d6e9de61a None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:47.153259 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d8bc7e-1a72-483f-bff7-500d6e9de61a None admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:47.154074 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d8bc7e-1a72-483f-bff7-500d6e9de61a None admin] RBAC: Adding query filter params (name=service) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:47.154869 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d8bc7e-1a72-483f-bff7-500d6e9de61a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:47.156838 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d8bc7e-1a72-483f-bff7-500d6e9de61a None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:47.166601 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 225/453] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Oct 31 12:20:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:47.166601 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d8bc7e-1a72-483f-bff7-500d6e9de61a None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:47.255030 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:47.273608 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592ac4ca-f6dd-4c13-8db0-60136577c014 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:47.284025 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d8bc7e-1a72-483f-bff7-500d6e9de61a None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:47.294977 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:47.314938 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2489e53-7d0d-4235-becb-43cfd8654f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:47.315977 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2489e53-7d0d-4235-becb-43cfd8654f87 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:47.315977 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2489e53-7d0d-4235-becb-43cfd8654f87 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:47.316140 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2489e53-7d0d-4235-becb-43cfd8654f87 None admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:47.317487 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2489e53-7d0d-4235-becb-43cfd8654f87 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:47.317487 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2489e53-7d0d-4235-becb-43cfd8654f87 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:47.319246 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2489e53-7d0d-4235-becb-43cfd8654f87 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:47.329134 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e6c546-ccb4-41f2-a3b6-4bce39a7c5cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:47.329767 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e6c546-ccb4-41f2-a3b6-4bce39a7c5cf None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:47.330041 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e6c546-ccb4-41f2-a3b6-4bce39a7c5cf None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:47.330308 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e6c546-ccb4-41f2-a3b6-4bce39a7c5cf None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:47.338892 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e6c546-ccb4-41f2-a3b6-4bce39a7c5cf None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:47.339586 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 229/454] 10.222.0.15 () {56 vars in 1133 bytes} [Mon Oct 31 12:20:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:47.340363 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e6c546-ccb4-41f2-a3b6-4bce39a7c5cf None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:47.340476 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2489e53-7d0d-4235-becb-43cfd8654f87 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:47.355109 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-c6e6c546-ccb4-41f2-a3b6-4bce39a7c5cf None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 31 12:20:47.355109 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:47.355109 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:47.355109 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:47.355109 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:47.355109 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:47.355109 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:47.355109 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:47.355109 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:47.355109 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:47.355109 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:47.355109 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:47.355109 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 31 12:20:47.355109 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 31 12:20:47.355109 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 31 12:20:47.355109 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 31 12:20:47.356228 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:47.356228 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:47.356228 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 31 12:20:47.356228 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 31 12:20:47.356228 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 31 12:20:47.356228 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 31 12:20:47.356228 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:47.356228 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:47.356228 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:47.356228 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:47.356228 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:47.356228 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:47.356228 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:47.356228 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:47.356228 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:47.356228 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:47.356228 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:47.356228 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:47.357122 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:47.357122 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:47.357122 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 31 12:20:47.357122 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 31 12:20:47.357122 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 31 12:20:47.357122 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 31 12:20:47.357122 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 31 12:20:47.357122 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 31 12:20:47.357122 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 31 12:20:47.357122 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:20:47.357122 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 226/455] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Oct 31 12:20:47 2022] GET /identity/v3/users/cinder => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:20:47.357670 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e6c546-ccb4-41f2-a3b6-4bce39a7c5cf None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:47.451149 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2489e53-7d0d-4235-becb-43cfd8654f87 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:47.463654 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e6c546-ccb4-41f2-a3b6-4bce39a7c5cf None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:47.492636 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d00c02-b043-4303-9a64-ddc69c775533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:47.493400 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d00c02-b043-4303-9a64-ddc69c775533 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:47.493818 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d00c02-b043-4303-9a64-ddc69c775533 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:47.494279 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d00c02-b043-4303-9a64-ddc69c775533 None admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:47.495044 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d00c02-b043-4303-9a64-ddc69c775533 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:47.495442 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d00c02-b043-4303-9a64-ddc69c775533 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:47.496871 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d00c02-b043-4303-9a64-ddc69c775533 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:47.498105 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb089d0f-5e64-40f7-9e55-18f569b0ef36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:47.498813 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb089d0f-5e64-40f7-9e55-18f569b0ef36 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:47.499156 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb089d0f-5e64-40f7-9e55-18f569b0ef36 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:47.499524 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb089d0f-5e64-40f7-9e55-18f569b0ef36 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:47.514033 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 230/456] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Oct 31 12:20:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:47.514033 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d00c02-b043-4303-9a64-ddc69c775533 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:47.515743 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb089d0f-5e64-40f7-9e55-18f569b0ef36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:47.517259 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb089d0f-5e64-40f7-9e55-18f569b0ef36 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:47.522545 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:47.532558 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-fb089d0f-5e64-40f7-9e55-18f569b0ef36 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:20:47.532558 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:47.532558 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:47.532558 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:47.532558 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:47.532558 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:47.532558 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:47.532558 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:47.532558 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:47.532558 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:47.532558 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:47.532558 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:47.532558 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 31 12:20:47.532558 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 31 12:20:47.532558 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 31 12:20:47.532558 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 31 12:20:47.533642 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:47.533642 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:47.533642 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:47.533642 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:47.533642 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:47.533642 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:47.533642 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:47.533642 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:47.533642 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:47.533642 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:47.533642 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:47.533642 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:47.533642 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:47.533642 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:47.533642 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:47.533642 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:47.533642 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 31 12:20:47.534689 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 31 12:20:47.534689 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:20:47.534689 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:20:47.539848 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 227/457] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Oct 31 12:20:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:20:47.540679 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb089d0f-5e64-40f7-9e55-18f569b0ef36 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:47.568352 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:47.623490 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d00c02-b043-4303-9a64-ddc69c775533 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:47.653116 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb089d0f-5e64-40f7-9e55-18f569b0ef36 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:47.659435 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479c492a-294a-4721-a3eb-ca1a99291375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:47.660457 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479c492a-294a-4721-a3eb-ca1a99291375 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:47.660741 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479c492a-294a-4721-a3eb-ca1a99291375 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:47.661014 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479c492a-294a-4721-a3eb-ca1a99291375 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:47.688648 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac05a188-7d22-4a76-bc13-a6829c62b9bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:47.689249 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac05a188-7d22-4a76-bc13-a6829c62b9bf None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:47.689553 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac05a188-7d22-4a76-bc13-a6829c62b9bf None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:47.689789 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac05a188-7d22-4a76-bc13-a6829c62b9bf None admin] PATH_INFO: `/v3/users` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:47.690503 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac05a188-7d22-4a76-bc13-a6829c62b9bf None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:47.690834 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac05a188-7d22-4a76-bc13-a6829c62b9bf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:47.692282 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac05a188-7d22-4a76-bc13-a6829c62b9bf None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:47.712920 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 228/458] 10.222.0.15 () {56 vars in 1137 bytes} [Mon Oct 31 12:20:47 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:47.714541 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac05a188-7d22-4a76-bc13-a6829c62b9bf None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:47.736620 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479c492a-294a-4721-a3eb-ca1a99291375 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:47.738064 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479c492a-294a-4721-a3eb-ca1a99291375 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:47.746941 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:47.799589 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:47.823393 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac05a188-7d22-4a76-bc13-a6829c62b9bf None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:47.846577 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 231/459] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Oct 31 12:20:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 31 12:20:47.852007 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479c492a-294a-4721-a3eb-ca1a99291375 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:47.857621 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a7a042-dd13-4ec2-95a4-df299bf767af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:47.858252 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a7a042-dd13-4ec2-95a4-df299bf767af None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:47.858531 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a7a042-dd13-4ec2-95a4-df299bf767af None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:47.858800 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a7a042-dd13-4ec2-95a4-df299bf767af None admin] PATH_INFO: `/v3/users/glance` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:47.866046 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a7a042-dd13-4ec2-95a4-df299bf767af None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:47.867529 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a7a042-dd13-4ec2-95a4-df299bf767af None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:47.880442 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-64a7a042-dd13-4ec2-95a4-df299bf767af None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 31 12:20:47.880442 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:47.880442 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:47.880442 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:47.880442 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:47.880442 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:47.880442 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:47.880442 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:47.880442 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:47.880442 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:47.880442 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:47.880442 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:47.880442 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 31 12:20:47.880442 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 31 12:20:47.880442 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 31 12:20:47.880442 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 31 12:20:47.881541 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:47.881541 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:47.881541 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 31 12:20:47.881541 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 31 12:20:47.881541 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 31 12:20:47.881541 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 31 12:20:47.881541 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:47.881541 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:47.881541 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:47.881541 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:47.881541 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:47.881541 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:47.881541 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:47.881541 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:47.881541 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:47.881541 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:47.881541 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:47.881541 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:47.882426 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:47.882426 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:47.882426 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 31 12:20:47.882426 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 31 12:20:47.882426 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 31 12:20:47.882426 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 31 12:20:47.882426 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 31 12:20:47.882426 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 31 12:20:47.882426 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 31 12:20:47.882426 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:20:47.882426 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 229/460] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Oct 31 12:20:47 2022] GET /identity/v3/users/glance => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:20:47.882979 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a7a042-dd13-4ec2-95a4-df299bf767af None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:47.965103 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479c492a-294a-4721-a3eb-ca1a99291375 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:47.973112 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:47.984574 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a7a042-dd13-4ec2-95a4-df299bf767af None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:48.003045 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48042b17-06a1-4c54-ba46-5c0d34a5568f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:48.003682 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48042b17-06a1-4c54-ba46-5c0d34a5568f None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:48.003961 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48042b17-06a1-4c54-ba46-5c0d34a5568f None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:48.004226 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48042b17-06a1-4c54-ba46-5c0d34a5568f None admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:48.004853 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48042b17-06a1-4c54-ba46-5c0d34a5568f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:48.005409 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48042b17-06a1-4c54-ba46-5c0d34a5568f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:48.006495 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48042b17-06a1-4c54-ba46-5c0d34a5568f None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:48.019510 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 232/461] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Oct 31 12:20:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:48.020471 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48042b17-06a1-4c54-ba46-5c0d34a5568f None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:48.028502 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00031d1-26eb-452f-a808-2cb1490ab26e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:48.029142 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00031d1-26eb-452f-a808-2cb1490ab26e None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:48.029416 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00031d1-26eb-452f-a808-2cb1490ab26e None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:48.029684 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00031d1-26eb-452f-a808-2cb1490ab26e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:48.040593 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:48.104065 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00031d1-26eb-452f-a808-2cb1490ab26e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:48.105534 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00031d1-26eb-452f-a808-2cb1490ab26e None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:48.135447 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48042b17-06a1-4c54-ba46-5c0d34a5568f None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:48.171373 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87cd663-8fb3-4bbf-85a2-c7207f1d9fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:48.172311 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87cd663-8fb3-4bbf-85a2-c7207f1d9fd1 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:48.172780 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87cd663-8fb3-4bbf-85a2-c7207f1d9fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:48.173215 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87cd663-8fb3-4bbf-85a2-c7207f1d9fd1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:48.184638 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87cd663-8fb3-4bbf-85a2-c7207f1d9fd1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:48.186264 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87cd663-8fb3-4bbf-85a2-c7207f1d9fd1 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:48.201867 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-e87cd663-8fb3-4bbf-85a2-c7207f1d9fd1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:20:48.201867 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:48.201867 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:48.201867 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:48.201867 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:48.201867 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:48.201867 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:48.201867 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:48.201867 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:48.201867 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:48.201867 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:48.201867 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:48.201867 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 31 12:20:48.201867 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 31 12:20:48.201867 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 31 12:20:48.201867 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 31 12:20:48.202744 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:48.202744 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:48.202744 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:48.202744 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:48.202744 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:48.202744 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:48.202744 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:48.202744 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:48.202744 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:48.202744 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:48.202744 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:48.202744 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:48.202744 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:48.202744 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:48.202744 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:48.202744 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:48.202744 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 31 12:20:48.203611 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 31 12:20:48.203611 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:20:48.203611 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:20:48.205282 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 233/462] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Oct 31 12:20:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:20:48.206230 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87cd663-8fb3-4bbf-85a2-c7207f1d9fd1 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:48.214187 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 230/463] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Oct 31 12:20:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 31 12:20:48.215027 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00031d1-26eb-452f-a808-2cb1490ab26e None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:48.230114 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:48.257727 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:48.317787 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87cd663-8fb3-4bbf-85a2-c7207f1d9fd1 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:48.324227 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00031d1-26eb-452f-a808-2cb1490ab26e None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:48.352867 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b10a9a-f67a-4940-aea0-9bc5898ee2c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:48.353471 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b10a9a-f67a-4940-aea0-9bc5898ee2c2 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:48.354494 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b10a9a-f67a-4940-aea0-9bc5898ee2c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:48.354763 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b10a9a-f67a-4940-aea0-9bc5898ee2c2 None admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:48.355511 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b10a9a-f67a-4940-aea0-9bc5898ee2c2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:48.355815 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b10a9a-f67a-4940-aea0-9bc5898ee2c2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:48.357249 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b10a9a-f67a-4940-aea0-9bc5898ee2c2 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:48.359335 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d40520-6ad1-467d-b2cf-89a38bcd6d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:48.359966 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d40520-6ad1-467d-b2cf-89a38bcd6d1c None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:48.360247 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d40520-6ad1-467d-b2cf-89a38bcd6d1c None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:48.360516 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d40520-6ad1-467d-b2cf-89a38bcd6d1c None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:48.367778 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d40520-6ad1-467d-b2cf-89a38bcd6d1c None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:48.369312 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d40520-6ad1-467d-b2cf-89a38bcd6d1c None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:48.375671 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 234/464] 10.222.0.15 () {56 vars in 1137 bytes} [Mon Oct 31 12:20:48 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:48.376330 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b10a9a-f67a-4940-aea0-9bc5898ee2c2 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:48.387098 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-91d40520-6ad1-467d-b2cf-89a38bcd6d1c None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 31 12:20:48.387098 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:48.387098 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:48.387098 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:48.387098 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:48.387098 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:48.387098 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:48.387098 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:48.387098 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:48.387098 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:48.387098 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:48.387098 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:48.387098 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 31 12:20:48.387098 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 31 12:20:48.387098 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 31 12:20:48.387098 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 31 12:20:48.388244 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:48.388244 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:48.388244 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 31 12:20:48.388244 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 31 12:20:48.388244 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 31 12:20:48.388244 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 31 12:20:48.388244 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:48.388244 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:48.388244 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:48.388244 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:48.388244 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:48.388244 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:48.388244 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:48.388244 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:48.388244 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:48.388244 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:48.388244 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:48.388244 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:48.389167 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:48.389167 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:48.389167 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 31 12:20:48.389167 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 31 12:20:48.389167 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 31 12:20:48.389167 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 31 12:20:48.389167 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 31 12:20:48.389167 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 31 12:20:48.389167 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 31 12:20:48.389167 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:20:48.389167 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 231/465] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Oct 31 12:20:48 2022] GET /identity/v3/users/neutron => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:20:48.389734 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d40520-6ad1-467d-b2cf-89a38bcd6d1c None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:48.470533 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:48.496079 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b10a9a-f67a-4940-aea0-9bc5898ee2c2 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:48.501928 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:48.505809 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d40520-6ad1-467d-b2cf-89a38bcd6d1c None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:48.531924 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d571a44-2898-4572-bbf4-41b07b1dacb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:48.532530 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d571a44-2898-4572-bbf4-41b07b1dacb4 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:48.532808 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d571a44-2898-4572-bbf4-41b07b1dacb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:48.533075 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d571a44-2898-4572-bbf4-41b07b1dacb4 None admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:48.533716 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d571a44-2898-4572-bbf4-41b07b1dacb4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:48.534007 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d571a44-2898-4572-bbf4-41b07b1dacb4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:48.535379 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d571a44-2898-4572-bbf4-41b07b1dacb4 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:48.551623 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 235/466] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Oct 31 12:20:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:48.551772 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1644b51-408c-4b74-85c7-764304063436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:48.551900 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d571a44-2898-4572-bbf4-41b07b1dacb4 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:48.551986 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1644b51-408c-4b74-85c7-764304063436 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:48.551986 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1644b51-408c-4b74-85c7-764304063436 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:48.551986 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1644b51-408c-4b74-85c7-764304063436 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:48.563419 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1644b51-408c-4b74-85c7-764304063436 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:48.564895 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1644b51-408c-4b74-85c7-764304063436 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:48.579838 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-c1644b51-408c-4b74-85c7-764304063436 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:20:48.579838 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:48.579838 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:48.579838 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:48.579838 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:48.579838 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:48.579838 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:48.579838 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:48.579838 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:48.579838 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:48.579838 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:48.579838 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:48.579838 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 31 12:20:48.579838 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 31 12:20:48.579838 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 31 12:20:48.579838 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 31 12:20:48.580720 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:48.580720 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:48.580720 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:48.580720 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:48.580720 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:48.580720 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:48.580720 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:48.580720 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:48.580720 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:48.580720 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:48.580720 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:48.580720 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:48.580720 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:48.580720 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:48.580720 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:48.580720 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:48.580720 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 31 12:20:48.581585 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 31 12:20:48.581585 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:20:48.581585 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:20:48.581585 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 232/467] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Oct 31 12:20:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:20:48.581802 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1644b51-408c-4b74-85c7-764304063436 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:48.665282 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d571a44-2898-4572-bbf4-41b07b1dacb4 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:48.690206 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1644b51-408c-4b74-85c7-764304063436 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:48.704170 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4036e74e-046b-4c3e-b60a-b9dc11ffb158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:48.705148 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4036e74e-046b-4c3e-b60a-b9dc11ffb158 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:48.705427 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4036e74e-046b-4c3e-b60a-b9dc11ffb158 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:48.705968 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4036e74e-046b-4c3e-b60a-b9dc11ffb158 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:48.710216 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:48.725115 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd46ffc1-dfbc-499e-ba6e-bbd356937e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:48.725801 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd46ffc1-dfbc-499e-ba6e-bbd356937e05 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:48.726067 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd46ffc1-dfbc-499e-ba6e-bbd356937e05 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:48.726253 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd46ffc1-dfbc-499e-ba6e-bbd356937e05 None admin] PATH_INFO: `/v3/users` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:48.726973 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd46ffc1-dfbc-499e-ba6e-bbd356937e05 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:48.727315 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd46ffc1-dfbc-499e-ba6e-bbd356937e05 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:48.728776 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd46ffc1-dfbc-499e-ba6e-bbd356937e05 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:48.746932 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 233/468] 10.222.0.15 () {56 vars in 1139 bytes} [Mon Oct 31 12:20:48 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:48.747699 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd46ffc1-dfbc-499e-ba6e-bbd356937e05 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:48.758526 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:48.785556 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4036e74e-046b-4c3e-b60a-b9dc11ffb158 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:48.787300 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4036e74e-046b-4c3e-b60a-b9dc11ffb158 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:48.858389 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd46ffc1-dfbc-499e-ba6e-bbd356937e05 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:48.895784 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 236/469] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Oct 31 12:20:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 31 12:20:48.896105 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435c379a-76ac-4af9-8464-8ecec0a24210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:48.896611 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4036e74e-046b-4c3e-b60a-b9dc11ffb158 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:48.897171 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435c379a-76ac-4af9-8464-8ecec0a24210 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:48.897645 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435c379a-76ac-4af9-8464-8ecec0a24210 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:48.898128 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435c379a-76ac-4af9-8464-8ecec0a24210 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:48.905860 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435c379a-76ac-4af9-8464-8ecec0a24210 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:48.907322 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435c379a-76ac-4af9-8464-8ecec0a24210 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:48.918967 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-435c379a-76ac-4af9-8464-8ecec0a24210 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 31 12:20:48.918967 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:48.918967 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:48.918967 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:48.918967 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:48.918967 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:48.918967 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:48.918967 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:48.918967 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:48.918967 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:48.918967 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:48.918967 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:48.918967 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 31 12:20:48.918967 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 31 12:20:48.918967 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 31 12:20:48.919980 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 31 12:20:48.919980 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:48.919980 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:48.919980 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:48.919980 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:48.919980 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:48.919980 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:48.919980 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:48.919980 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:48.919980 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:48.919980 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:48.919980 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:48.919980 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:48.919980 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:48.919980 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:48.919980 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:48.919980 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:48.921312 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 31 12:20:48.921312 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 31 12:20:48.921312 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 31 12:20:48.921312 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 31 12:20:48.921312 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 31 12:20:48.921312 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 31 12:20:48.921312 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 31 12:20:48.921312 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:20:48.921312 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 234/470] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Oct 31 12:20:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:20:48.921312 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435c379a-76ac-4af9-8464-8ecec0a24210 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:48.940205 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:48.991049 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:49.015156 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4036e74e-046b-4c3e-b60a-b9dc11ffb158 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:49.033176 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435c379a-76ac-4af9-8464-8ecec0a24210 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:49.053016 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3999cd8-18a0-4875-826e-9258524c6ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:49.053746 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3999cd8-18a0-4875-826e-9258524c6ef8 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:49.054024 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3999cd8-18a0-4875-826e-9258524c6ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:49.054294 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3999cd8-18a0-4875-826e-9258524c6ef8 None admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:49.054929 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3999cd8-18a0-4875-826e-9258524c6ef8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:49.055260 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3999cd8-18a0-4875-826e-9258524c6ef8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:49.057406 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3999cd8-18a0-4875-826e-9258524c6ef8 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:49.067524 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e318cdb9-d6d6-4cfe-8d2b-c1aed88b335e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:49.068176 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e318cdb9-d6d6-4cfe-8d2b-c1aed88b335e None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:49.068455 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e318cdb9-d6d6-4cfe-8d2b-c1aed88b335e None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:49.068720 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e318cdb9-d6d6-4cfe-8d2b-c1aed88b335e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:49.070997 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 237/471] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Oct 31 12:20:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:49.072801 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3999cd8-18a0-4875-826e-9258524c6ef8 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:49.143297 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e318cdb9-d6d6-4cfe-8d2b-c1aed88b335e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:49.145059 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e318cdb9-d6d6-4cfe-8d2b-c1aed88b335e None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:49.172811 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:49.179778 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3999cd8-18a0-4875-826e-9258524c6ef8 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:49.220562 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fe639f-45d9-4504-8181-4dfc32898672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:49.221199 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fe639f-45d9-4504-8181-4dfc32898672 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:49.221467 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fe639f-45d9-4504-8181-4dfc32898672 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:49.221671 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:49.221785 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fe639f-45d9-4504-8181-4dfc32898672 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:49.232687 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fe639f-45d9-4504-8181-4dfc32898672 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:49.234129 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fe639f-45d9-4504-8181-4dfc32898672 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:49.249363 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-d6fe639f-45d9-4504-8181-4dfc32898672 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:20:49.249363 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:49.249363 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:49.249363 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:49.249363 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:49.249363 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:49.249363 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:49.249363 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:49.249363 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:49.249363 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:49.249363 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:49.249363 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:49.249363 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 31 12:20:49.249363 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 31 12:20:49.249363 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 31 12:20:49.249363 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 31 12:20:49.250288 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:49.250288 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:49.250288 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:49.250288 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:49.250288 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:49.250288 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:49.250288 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:49.250288 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:49.250288 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:49.250288 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:49.250288 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:49.250288 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:49.250288 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:49.250288 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:49.250288 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:49.250288 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:49.250288 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 31 12:20:49.251124 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 31 12:20:49.251124 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:20:49.251124 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:20:49.251124 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 238/472] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Oct 31 12:20:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:20:49.251821 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fe639f-45d9-4504-8181-4dfc32898672 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:49.265534 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 235/473] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Oct 31 12:20:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 31 12:20:49.266705 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e318cdb9-d6d6-4cfe-8d2b-c1aed88b335e None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:49.361609 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fe639f-45d9-4504-8181-4dfc32898672 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:49.370340 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e318cdb9-d6d6-4cfe-8d2b-c1aed88b335e None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:49.395475 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f3aa03-7d61-4980-b9c2-76da573437ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:49.396094 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f3aa03-7d61-4980-b9c2-76da573437ec None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:49.396370 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f3aa03-7d61-4980-b9c2-76da573437ec None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:49.396645 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f3aa03-7d61-4980-b9c2-76da573437ec None admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:49.397311 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f3aa03-7d61-4980-b9c2-76da573437ec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:49.397607 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f3aa03-7d61-4980-b9c2-76da573437ec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:49.399014 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f3aa03-7d61-4980-b9c2-76da573437ec None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:49.405074 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef32d722-1974-40bc-b45a-993ede68721c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:49.405695 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef32d722-1974-40bc-b45a-993ede68721c None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:49.405974 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef32d722-1974-40bc-b45a-993ede68721c None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:49.406246 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef32d722-1974-40bc-b45a-993ede68721c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:49.411266 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 239/474] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Oct 31 12:20:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:49.412070 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f3aa03-7d61-4980-b9c2-76da573437ec None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:49.413224 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef32d722-1974-40bc-b45a-993ede68721c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:49.414655 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef32d722-1974-40bc-b45a-993ede68721c None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:49.422554 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:49.427154 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-ef32d722-1974-40bc-b45a-993ede68721c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 31 12:20:49.427154 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:49.427154 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:49.427154 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:49.427154 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:49.427154 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:49.427154 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:49.427154 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:49.427154 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:49.427154 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:49.427154 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:49.427154 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:49.427154 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 31 12:20:49.427154 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 31 12:20:49.427154 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 31 12:20:49.428170 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 31 12:20:49.428170 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:49.428170 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:49.428170 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:49.428170 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:49.428170 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:49.428170 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:49.428170 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:49.428170 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:49.428170 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:49.428170 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:49.428170 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:49.428170 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:49.428170 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:49.428170 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:49.428170 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:49.428170 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:49.429003 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 31 12:20:49.429003 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 31 12:20:49.429003 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 31 12:20:49.429003 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 31 12:20:49.429003 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 31 12:20:49.429003 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 31 12:20:49.429003 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 31 12:20:49.429003 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:20:49.429003 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 236/475] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Oct 31 12:20:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:20:49.429003 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef32d722-1974-40bc-b45a-993ede68721c None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:49.460130 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:49.514088 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f3aa03-7d61-4980-b9c2-76da573437ec None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:49.531745 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef32d722-1974-40bc-b45a-993ede68721c None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:49.549442 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a63eda8-6ad5-4147-8389-f18909ddad6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:49.550043 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a63eda8-6ad5-4147-8389-f18909ddad6d None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:49.550323 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a63eda8-6ad5-4147-8389-f18909ddad6d None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:49.550590 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a63eda8-6ad5-4147-8389-f18909ddad6d None admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:49.551247 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a63eda8-6ad5-4147-8389-f18909ddad6d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:49.551554 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a63eda8-6ad5-4147-8389-f18909ddad6d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:49.552902 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a63eda8-6ad5-4147-8389-f18909ddad6d None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:49.564777 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 240/476] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Oct 31 12:20:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:49.565241 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d3641a-2e48-4e8c-94ea-66f9c1b1c369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:49.565939 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e34dfe-5ebb-42ff-910b-6e0cbe338bff None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:49.566038 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d3641a-2e48-4e8c-94ea-66f9c1b1c369 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:49.566226 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e34dfe-5ebb-42ff-910b-6e0cbe338bff None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:49.566327 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d3641a-2e48-4e8c-94ea-66f9c1b1c369 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:49.566497 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d3641a-2e48-4e8c-94ea-66f9c1b1c369 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:49.566588 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e34dfe-5ebb-42ff-910b-6e0cbe338bff None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:49.571419 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 241/477] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:20:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:20:49.571419 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e34dfe-5ebb-42ff-910b-6e0cbe338bff None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:49.580388 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d3641a-2e48-4e8c-94ea-66f9c1b1c369 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:49.581828 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d3641a-2e48-4e8c-94ea-66f9c1b1c369 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:49.596646 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-c1d3641a-2e48-4e8c-94ea-66f9c1b1c369 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:20:49.596646 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:49.596646 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:49.596646 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:49.596646 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:49.596646 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:49.596646 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:49.596646 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:49.596646 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:49.596646 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:49.596646 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:49.596646 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:49.596646 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 31 12:20:49.596646 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 31 12:20:49.596646 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 31 12:20:49.596646 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 31 12:20:49.597742 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:49.597742 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:49.597742 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:49.597742 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:49.597742 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:49.597742 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:49.597742 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:49.597742 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:49.597742 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:49.597742 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:49.597742 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:49.597742 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:49.597742 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:49.597742 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:49.597742 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:49.597742 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:49.597742 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 31 12:20:49.598610 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 31 12:20:49.598610 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:20:49.598610 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:20:49.598610 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 237/478] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Oct 31 12:20:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:20:49.598610 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d3641a-2e48-4e8c-94ea-66f9c1b1c369 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:49.642027 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:49.677777 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:49.677991 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e34dfe-5ebb-42ff-910b-6e0cbe338bff None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:49.698701 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d3641a-2e48-4e8c-94ea-66f9c1b1c369 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:49.711317 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b325c2fa-ca38-4028-8e0b-585f97fdcf9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:49.711918 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b325c2fa-ca38-4028-8e0b-585f97fdcf9d None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:49.712195 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b325c2fa-ca38-4028-8e0b-585f97fdcf9d None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:49.712463 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b325c2fa-ca38-4028-8e0b-585f97fdcf9d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:49.713769 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b325c2fa-ca38-4028-8e0b-585f97fdcf9d None admin] RBAC: Adding query filter params (role.id=9674a2c5acc74dfeb93cfd67dfd890dd, scope.project.id=2c32444cef584bce966b38203ce128b2, user.id=99889b1dab0b471d9c77887cffefc652) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:49.714076 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b325c2fa-ca38-4028-8e0b-585f97fdcf9d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:49.715581 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b325c2fa-ca38-4028-8e0b-585f97fdcf9d None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:49.721172 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 242/479] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Oct 31 12:20:49 2022] GET /identity/v3/role_assignments?user.id=99889b1dab0b471d9c77887cffefc652&scope.project.id=2c32444cef584bce966b38203ce128b2&role.id=9674a2c5acc74dfeb93cfd67dfd890dd => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:49.721944 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b325c2fa-ca38-4028-8e0b-585f97fdcf9d None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:49.735301 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ced7a3-7b8f-4527-af16-28f1e1a8b3c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:49.736030 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ced7a3-7b8f-4527-af16-28f1e1a8b3c9 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:49.736318 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ced7a3-7b8f-4527-af16-28f1e1a8b3c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:49.736571 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ced7a3-7b8f-4527-af16-28f1e1a8b3c9 None admin] PATH_INFO: `/v3/projects` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:49.737269 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ced7a3-7b8f-4527-af16-28f1e1a8b3c9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:49.737573 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ced7a3-7b8f-4527-af16-28f1e1a8b3c9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:49.739302 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ced7a3-7b8f-4527-af16-28f1e1a8b3c9 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:49.753007 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 238/480] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Oct 31 12:20:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:49.759402 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04644f7-adea-4393-b047-3c9fc1286009 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:49.759700 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04644f7-adea-4393-b047-3c9fc1286009 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:49.760442 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04644f7-adea-4393-b047-3c9fc1286009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:49.780545 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-c04644f7-adea-4393-b047-3c9fc1286009 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:20:49.781749 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-c04644f7-adea-4393-b047-3c9fc1286009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:20:49.837758 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b325c2fa-ca38-4028-8e0b-585f97fdcf9d None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:49.863202 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 239/481] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:20:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 31 12:20:49.863202 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04644f7-adea-4393-b047-3c9fc1286009 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:49.883409 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:49.893898 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b67cf2-3386-4215-a41d-55fa19f6bb09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:49.894554 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b67cf2-3386-4215-a41d-55fa19f6bb09 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:49.894833 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b67cf2-3386-4215-a41d-55fa19f6bb09 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:49.895105 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b67cf2-3386-4215-a41d-55fa19f6bb09 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:49.905200 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b67cf2-3386-4215-a41d-55fa19f6bb09 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:49.907222 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b67cf2-3386-4215-a41d-55fa19f6bb09 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:49.922322 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-60b67cf2-3386-4215-a41d-55fa19f6bb09 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 31 12:20:49.922322 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:49.922322 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:49.922322 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:49.922322 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:49.922322 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:49.922322 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:49.922322 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:49.922322 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:49.922322 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:49.922322 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:49.922322 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:49.922322 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 31 12:20:49.922322 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 31 12:20:49.922322 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 31 12:20:49.923443 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 31 12:20:49.923443 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:49.923443 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:49.923443 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:49.923443 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:49.923443 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:49.923443 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:49.923443 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:49.923443 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:49.923443 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:49.923443 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:49.923443 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:49.923443 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:49.923443 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:49.923443 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:49.923443 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:49.923443 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:49.924330 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 31 12:20:49.924330 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 31 12:20:49.924330 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 31 12:20:49.924330 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 31 12:20:49.924330 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 31 12:20:49.924330 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 31 12:20:49.924330 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 31 12:20:49.924330 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:20:49.924330 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 243/482] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Oct 31 12:20:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:20:49.924790 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b67cf2-3386-4215-a41d-55fa19f6bb09 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:49.946462 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:49.990744 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04644f7-adea-4393-b047-3c9fc1286009 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:50.035047 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5e967d-9bd5-4bf4-9ec6-9701944db9af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:50.035740 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5e967d-9bd5-4bf4-9ec6-9701944db9af None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:50.035984 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5e967d-9bd5-4bf4-9ec6-9701944db9af None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:50.036265 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5e967d-9bd5-4bf4-9ec6-9701944db9af None admin] PATH_INFO: `/v3/domains` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:50.036916 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5e967d-9bd5-4bf4-9ec6-9701944db9af None admin] RBAC: Adding query filter params (name=Default) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:50.037386 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5e967d-9bd5-4bf4-9ec6-9701944db9af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:50.038766 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5e967d-9bd5-4bf4-9ec6-9701944db9af None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:50.051457 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 240/483] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Oct 31 12:20:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:50.052290 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5e967d-9bd5-4bf4-9ec6-9701944db9af None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:50.058016 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b67cf2-3386-4215-a41d-55fa19f6bb09 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:50.096929 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b69f30e-743d-4484-9432-bde2220e4a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:50.097885 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b69f30e-743d-4484-9432-bde2220e4a7b None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:50.098352 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b69f30e-743d-4484-9432-bde2220e4a7b None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:50.098844 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b69f30e-743d-4484-9432-bde2220e4a7b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:50.099857 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b69f30e-743d-4484-9432-bde2220e4a7b None admin] RBAC: Adding query filter params (role.id=9674a2c5acc74dfeb93cfd67dfd890dd, scope.project.id=2c32444cef584bce966b38203ce128b2, user.id=20d6d7ea077d49e8a66c6c6762499702) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:50.100362 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b69f30e-743d-4484-9432-bde2220e4a7b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:50.102185 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b69f30e-743d-4484-9432-bde2220e4a7b None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:50.116976 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 244/484] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Oct 31 12:20:49 2022] GET /identity/v3/role_assignments?user.id=20d6d7ea077d49e8a66c6c6762499702&scope.project.id=2c32444cef584bce966b38203ce128b2&role.id=9674a2c5acc74dfeb93cfd67dfd890dd => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:50.118588 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b69f30e-743d-4484-9432-bde2220e4a7b None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:50.171097 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5e967d-9bd5-4bf4-9ec6-9701944db9af None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:50.203726 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:50.205720 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:50.214088 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b0ae5a-8e1f-4a45-9c20-ac0d4322f341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:50.215739 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b0ae5a-8e1f-4a45-9c20-ac0d4322f341 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:50.216326 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b0ae5a-8e1f-4a45-9c20-ac0d4322f341 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:50.216829 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b0ae5a-8e1f-4a45-9c20-ac0d4322f341 None admin] PATH_INFO: `/v3/projects` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:50.217658 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b0ae5a-8e1f-4a45-9c20-ac0d4322f341 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:50.218070 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b0ae5a-8e1f-4a45-9c20-ac0d4322f341 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:50.219711 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b0ae5a-8e1f-4a45-9c20-ac0d4322f341 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:50.238892 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 241/485] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Oct 31 12:20:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:50.240090 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b0ae5a-8e1f-4a45-9c20-ac0d4322f341 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:50.264347 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b69f30e-743d-4484-9432-bde2220e4a7b None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:50.308723 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633cb6e4-3de7-43f9-82ab-196d6135ccca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:50.309774 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633cb6e4-3de7-43f9-82ab-196d6135ccca None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:50.310517 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633cb6e4-3de7-43f9-82ab-196d6135ccca None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:50.311238 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633cb6e4-3de7-43f9-82ab-196d6135ccca None admin] PATH_INFO: `/v3/projects/service` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:50.322325 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633cb6e4-3de7-43f9-82ab-196d6135ccca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:50.324646 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633cb6e4-3de7-43f9-82ab-196d6135ccca None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:50.339493 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-633cb6e4-3de7-43f9-82ab-196d6135ccca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 31 12:20:50.339493 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:50.339493 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:50.339493 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:50.339493 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:50.339493 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:50.339493 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:50.339493 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:50.339493 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:50.339493 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:50.339493 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:50.339493 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:50.339493 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 31 12:20:50.339493 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 31 12:20:50.339493 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 31 12:20:50.340638 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 31 12:20:50.340638 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:50.340638 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:50.340638 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:50.340638 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:50.340638 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:50.340638 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:50.340638 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:50.340638 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:50.340638 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:50.340638 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:50.340638 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:50.340638 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:50.340638 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:50.340638 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:50.340638 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:50.340638 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:50.341688 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 31 12:20:50.341688 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 31 12:20:50.341688 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 31 12:20:50.341688 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 31 12:20:50.341688 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 31 12:20:50.341688 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 31 12:20:50.341688 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 31 12:20:50.341688 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:20:50.341688 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 245/486] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Oct 31 12:20:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:20:50.342278 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633cb6e4-3de7-43f9-82ab-196d6135ccca None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:50.368240 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b0ae5a-8e1f-4a45-9c20-ac0d4322f341 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:50.419532 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23195fce-6c54-4850-b5fb-1a85933654ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:50.421465 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23195fce-6c54-4850-b5fb-1a85933654ba None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:50.421894 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23195fce-6c54-4850-b5fb-1a85933654ba None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:50.422596 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23195fce-6c54-4850-b5fb-1a85933654ba None admin] PATH_INFO: `/v3/roles/service` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:50.439798 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23195fce-6c54-4850-b5fb-1a85933654ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 31 12:20:50.445286 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-23195fce-6c54-4850-b5fb-1a85933654ba None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 31 12:20:50.445286 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:50.445286 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:50.445286 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:50.445286 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:50.445286 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:50.445286 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:50.445286 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:50.445286 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:50.445286 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:50.445286 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:50.445286 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:50.445286 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 31 12:20:50.445286 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 31 12:20:50.445286 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 31 12:20:50.445286 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 31 12:20:50.446398 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 31 12:20:50.446398 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 31 12:20:50.446398 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 31 12:20:50.446398 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 31 12:20:50.446398 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:50.446398 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:50.446398 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:50.446398 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:50.446398 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:50.446398 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:50.446398 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:50.446398 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:50.446398 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:50.446398 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:50.446398 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:50.446398 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:50.448141 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:50.448141 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:50.448141 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:50.448141 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:50.448141 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 31 12:20:50.448141 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 31 12:20:50.448141 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 31 12:20:50.448141 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 31 12:20:50.448141 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 31 12:20:50.448141 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 31 12:20:50.448141 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 31 12:20:50.448141 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:20:50.449399 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 242/487] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Oct 31 12:20:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:20:50.450039 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23195fce-6c54-4850-b5fb-1a85933654ba None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:50.475300 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633cb6e4-3de7-43f9-82ab-196d6135ccca None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:50.493496 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:50.504682 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:50.515300 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8d7695-08e4-4d14-bada-3b5295549846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:50.516190 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8d7695-08e4-4d14-bada-3b5295549846 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:50.516676 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8d7695-08e4-4d14-bada-3b5295549846 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:50.517135 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8d7695-08e4-4d14-bada-3b5295549846 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:50.518044 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8d7695-08e4-4d14-bada-3b5295549846 None admin] RBAC: Adding query filter params (role.id=9674a2c5acc74dfeb93cfd67dfd890dd, scope.project.id=2c32444cef584bce966b38203ce128b2, user.id=4feaa01c5ed64b65b10d4dece823702c) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:50.518779 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8d7695-08e4-4d14-bada-3b5295549846 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:50.520406 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8d7695-08e4-4d14-bada-3b5295549846 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:50.535514 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 246/488] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Oct 31 12:20:50 2022] GET /identity/v3/role_assignments?user.id=4feaa01c5ed64b65b10d4dece823702c&scope.project.id=2c32444cef584bce966b38203ce128b2&role.id=9674a2c5acc74dfeb93cfd67dfd890dd => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:50.536813 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8d7695-08e4-4d14-bada-3b5295549846 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:50.564091 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23195fce-6c54-4850-b5fb-1a85933654ba None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:50.623497 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f442310-4ba1-4400-9104-8e7892a40939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:50.624124 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f442310-4ba1-4400-9104-8e7892a40939 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:50.624398 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f442310-4ba1-4400-9104-8e7892a40939 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:50.624665 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f442310-4ba1-4400-9104-8e7892a40939 None admin] PATH_INFO: `/v3/projects` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:50.625353 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f442310-4ba1-4400-9104-8e7892a40939 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:50.625650 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f442310-4ba1-4400-9104-8e7892a40939 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:50.627137 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f442310-4ba1-4400-9104-8e7892a40939 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:50.645629 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 243/489] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Oct 31 12:20:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:50.647040 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f442310-4ba1-4400-9104-8e7892a40939 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:50.669553 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8d7695-08e4-4d14-bada-3b5295549846 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:50.710756 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb6769d-dc62-4f0a-a672-6c85648ca615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:50.712243 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb6769d-dc62-4f0a-a672-6c85648ca615 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:50.712761 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb6769d-dc62-4f0a-a672-6c85648ca615 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:50.713959 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb6769d-dc62-4f0a-a672-6c85648ca615 None admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:50.714883 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb6769d-dc62-4f0a-a672-6c85648ca615 None admin] RBAC: Adding query filter params (name=service) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:50.715420 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb6769d-dc62-4f0a-a672-6c85648ca615 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:50.717016 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb6769d-dc62-4f0a-a672-6c85648ca615 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:50.726988 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 247/490] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Oct 31 12:20:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:50.734075 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb6769d-dc62-4f0a-a672-6c85648ca615 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:50.765024 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f442310-4ba1-4400-9104-8e7892a40939 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:50.778616 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:50.788191 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:50.807871 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a0680b-1927-4aab-8452-1b533a96b129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:50.808538 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a0680b-1927-4aab-8452-1b533a96b129 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:50.808761 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a0680b-1927-4aab-8452-1b533a96b129 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:50.809031 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a0680b-1927-4aab-8452-1b533a96b129 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:50.809742 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a0680b-1927-4aab-8452-1b533a96b129 None admin] RBAC: Adding query filter params (role.id=9674a2c5acc74dfeb93cfd67dfd890dd, scope.project.id=2c32444cef584bce966b38203ce128b2, user.id=25e9c7d1e4714bf18a1ced84855423d5) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:50.810056 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a0680b-1927-4aab-8452-1b533a96b129 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:50.811475 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a0680b-1927-4aab-8452-1b533a96b129 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:50.818035 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 244/491] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Oct 31 12:20:50 2022] GET /identity/v3/role_assignments?user.id=25e9c7d1e4714bf18a1ced84855423d5&scope.project.id=2c32444cef584bce966b38203ce128b2&role.id=9674a2c5acc74dfeb93cfd67dfd890dd => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:50.863903 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb6769d-dc62-4f0a-a672-6c85648ca615 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:50.908267 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af26cf5-1eac-4d7f-879e-4b63a7ece2e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:50.909155 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af26cf5-1eac-4d7f-879e-4b63a7ece2e4 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:50.909600 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af26cf5-1eac-4d7f-879e-4b63a7ece2e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:50.910013 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af26cf5-1eac-4d7f-879e-4b63a7ece2e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:50.924414 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af26cf5-1eac-4d7f-879e-4b63a7ece2e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:50.925898 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af26cf5-1eac-4d7f-879e-4b63a7ece2e4 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:50.944553 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-2af26cf5-1eac-4d7f-879e-4b63a7ece2e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:20:50.944553 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:50.944553 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:50.944553 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:50.944553 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:50.944553 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:50.944553 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:50.944553 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:50.944553 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:50.944553 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:50.944553 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:50.944553 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:50.944553 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 31 12:20:50.944553 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 31 12:20:50.944553 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 31 12:20:50.944553 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 31 12:20:50.945712 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:50.945712 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:50.945712 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:50.945712 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:50.945712 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:50.945712 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:50.945712 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:50.945712 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:50.945712 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:50.945712 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:50.945712 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:50.945712 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:50.945712 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:50.945712 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:50.945712 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:50.945712 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:50.945712 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 31 12:20:50.947389 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 31 12:20:50.947389 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:20:50.947389 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:20:50.947389 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 248/492] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Oct 31 12:20:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 31 12:20:50.952236 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a0680b-1927-4aab-8452-1b533a96b129 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:51.099321 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a0680b-1927-4aab-8452-1b533a96b129 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:51.136043 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4166ad8e-6407-411f-b0ac-0cafd2ee5093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:51.137187 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4166ad8e-6407-411f-b0ac-0cafd2ee5093 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:51.137704 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4166ad8e-6407-411f-b0ac-0cafd2ee5093 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:51.138202 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4166ad8e-6407-411f-b0ac-0cafd2ee5093 None admin] PATH_INFO: `/v3/domains` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:51.138988 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4166ad8e-6407-411f-b0ac-0cafd2ee5093 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:51.139456 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4166ad8e-6407-411f-b0ac-0cafd2ee5093 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:51.140967 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4166ad8e-6407-411f-b0ac-0cafd2ee5093 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:51.156403 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 245/493] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Oct 31 12:20:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:51.161642 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af26cf5-1eac-4d7f-879e-4b63a7ece2e4 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:51.300079 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:51.306541 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af26cf5-1eac-4d7f-879e-4b63a7ece2e4 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:51.355831 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9133e41-178f-4851-9e5e-5ea581e7c556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:51.356467 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9133e41-178f-4851-9e5e-5ea581e7c556 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:51.356801 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9133e41-178f-4851-9e5e-5ea581e7c556 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:51.357076 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9133e41-178f-4851-9e5e-5ea581e7c556 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:51.364507 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9133e41-178f-4851-9e5e-5ea581e7c556 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:51.365940 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9133e41-178f-4851-9e5e-5ea581e7c556 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:51.383292 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-f9133e41-178f-4851-9e5e-5ea581e7c556 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 31 12:20:51.383292 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:51.383292 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:51.383292 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:51.383292 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:51.383292 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:51.383292 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:51.383292 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:51.383292 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:51.383292 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:51.383292 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:51.383292 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:51.383292 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 31 12:20:51.383292 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 31 12:20:51.383292 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 31 12:20:51.383292 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 31 12:20:51.384233 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:51.384233 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:51.384233 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 31 12:20:51.384233 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 31 12:20:51.384233 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 31 12:20:51.384233 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 31 12:20:51.384233 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:51.384233 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:51.384233 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:51.384233 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:51.384233 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:51.384233 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:51.384233 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:51.384233 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:51.384233 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:51.384233 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:51.384233 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:51.384233 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:51.386303 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:51.386303 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:51.386303 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 31 12:20:51.386303 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 31 12:20:51.386303 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 31 12:20:51.386303 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 31 12:20:51.386303 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 31 12:20:51.386303 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 31 12:20:51.386303 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 31 12:20:51.386303 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:20:51.386303 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 249/494] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Oct 31 12:20:51 2022] GET /identity/v3/users/swift => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:20:51.390749 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4166ad8e-6407-411f-b0ac-0cafd2ee5093 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:51.458303 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:51.517611 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4166ad8e-6407-411f-b0ac-0cafd2ee5093 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:51.552286 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b5f67b-8715-41b2-8518-587c8e3a1699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:51.552811 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b5f67b-8715-41b2-8518-587c8e3a1699 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:51.553087 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b5f67b-8715-41b2-8518-587c8e3a1699 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:51.553353 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b5f67b-8715-41b2-8518-587c8e3a1699 None admin] PATH_INFO: `/v3/users` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:51.554072 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b5f67b-8715-41b2-8518-587c8e3a1699 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:51.554369 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b5f67b-8715-41b2-8518-587c8e3a1699 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:51.555818 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b5f67b-8715-41b2-8518-587c8e3a1699 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:51.586528 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 246/495] 10.222.0.15 () {56 vars in 1135 bytes} [Mon Oct 31 12:20:51 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:51.592681 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9133e41-178f-4851-9e5e-5ea581e7c556 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:51.727941 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9133e41-178f-4851-9e5e-5ea581e7c556 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:51.770904 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7011936-b190-4820-a498-eb562e7e7a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:51.771544 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7011936-b190-4820-a498-eb562e7e7a72 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:51.771822 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7011936-b190-4820-a498-eb562e7e7a72 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:51.772091 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7011936-b190-4820-a498-eb562e7e7a72 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:51.811671 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:51.859098 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7011936-b190-4820-a498-eb562e7e7a72 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:51.864611 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7011936-b190-4820-a498-eb562e7e7a72 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:51.983509 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 250/496] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Oct 31 12:20:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 31 12:20:51.988179 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b5f67b-8715-41b2-8518-587c8e3a1699 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:52.156257 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:52.165707 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b5f67b-8715-41b2-8518-587c8e3a1699 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:52.210608 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a71892-f650-42e6-93b7-0de06db4b6d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:52.211259 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a71892-f650-42e6-93b7-0de06db4b6d8 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:52.211533 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a71892-f650-42e6-93b7-0de06db4b6d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:52.211798 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a71892-f650-42e6-93b7-0de06db4b6d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:52.222215 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a71892-f650-42e6-93b7-0de06db4b6d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:52.231223 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a71892-f650-42e6-93b7-0de06db4b6d8 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:52.246897 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-49a71892-f650-42e6-93b7-0de06db4b6d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:20:52.246897 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:52.246897 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:52.246897 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:52.246897 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:52.246897 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:52.246897 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:52.246897 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:52.246897 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:52.246897 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:52.246897 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:52.246897 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:52.246897 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 31 12:20:52.246897 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 31 12:20:52.246897 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 31 12:20:52.246897 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 31 12:20:52.247950 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:52.247950 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:52.247950 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:52.247950 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:52.247950 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:52.247950 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:52.247950 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:52.247950 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:52.247950 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:52.247950 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:52.247950 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:52.247950 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:52.247950 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:52.247950 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:52.247950 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:52.247950 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:52.247950 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 31 12:20:52.248826 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 31 12:20:52.248826 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:20:52.248826 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:20:52.252284 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 247/497] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Oct 31 12:20:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:20:52.253603 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7011936-b190-4820-a498-eb562e7e7a72 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:52.336345 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:52.398062 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7011936-b190-4820-a498-eb562e7e7a72 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:52.442744 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc9f1c4-d09f-40ce-b0e7-20fcbf46a0ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:52.443717 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc9f1c4-d09f-40ce-b0e7-20fcbf46a0ee None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:52.444223 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc9f1c4-d09f-40ce-b0e7-20fcbf46a0ee None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:52.444691 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc9f1c4-d09f-40ce-b0e7-20fcbf46a0ee None admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:52.446141 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc9f1c4-d09f-40ce-b0e7-20fcbf46a0ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:52.446662 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc9f1c4-d09f-40ce-b0e7-20fcbf46a0ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:52.449474 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc9f1c4-d09f-40ce-b0e7-20fcbf46a0ee None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:52.462282 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 251/498] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Oct 31 12:20:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:52.468042 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a71892-f650-42e6-93b7-0de06db4b6d8 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:52.479427 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb117c3-88ca-4b07-9808-1b477bd7487d None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:52.479715 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb117c3-88ca-4b07-9808-1b477bd7487d None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:52.479991 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb117c3-88ca-4b07-9808-1b477bd7487d None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:52.481576 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 252/499] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:20:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:20:52.486415 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40843ef3-6434-455b-9d90-4b7d38284339 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:52.486695 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40843ef3-6434-455b-9d90-4b7d38284339 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:52.486967 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40843ef3-6434-455b-9d90-4b7d38284339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:52.517638 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-40843ef3-6434-455b-9d90-4b7d38284339 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:20:52.519060 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-40843ef3-6434-455b-9d90-4b7d38284339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:20:52.620875 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 253/500] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:20:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 31 12:20:52.634641 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a71892-f650-42e6-93b7-0de06db4b6d8 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:52.694575 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d262df-bad3-4def-bb16-24f2d6a432f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:52.695597 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d262df-bad3-4def-bb16-24f2d6a432f6 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:52.696106 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d262df-bad3-4def-bb16-24f2d6a432f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:52.696635 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d262df-bad3-4def-bb16-24f2d6a432f6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:52.707421 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d262df-bad3-4def-bb16-24f2d6a432f6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:52.709158 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d262df-bad3-4def-bb16-24f2d6a432f6 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:52.725055 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-d9d262df-bad3-4def-bb16-24f2d6a432f6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 31 12:20:52.725055 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:52.725055 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:52.725055 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:52.725055 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:52.725055 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:52.725055 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:52.725055 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:52.725055 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:52.725055 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:52.725055 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:52.725055 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:52.725055 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 31 12:20:52.725055 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 31 12:20:52.725055 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 31 12:20:52.726000 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 31 12:20:52.726000 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:52.726000 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:52.726000 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:52.726000 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:52.726000 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:52.726000 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:52.726000 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:52.726000 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:52.726000 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:52.726000 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:52.726000 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:52.726000 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:52.726000 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:52.726000 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:52.726000 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:52.726000 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:52.727362 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 31 12:20:52.727362 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 31 12:20:52.727362 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 31 12:20:52.727362 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 31 12:20:52.727362 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 31 12:20:52.727362 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 31 12:20:52.727362 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 31 12:20:52.727362 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:20:52.730800 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 248/501] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Oct 31 12:20:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:20:52.740251 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40843ef3-6434-455b-9d90-4b7d38284339 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:52.793596 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e19f932-451c-44dc-acc7-14873b17aa91 None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:52.794302 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e19f932-451c-44dc-acc7-14873b17aa91 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:52.794799 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e19f932-451c-44dc-acc7-14873b17aa91 None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:52.796582 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 249/502] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:20:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 31 12:20:52.803382 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1001a7-ba73-4c48-9e4a-5360198c8c5f None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:52.803667 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1001a7-ba73-4c48-9e4a-5360198c8c5f None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:52.804034 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1001a7-ba73-4c48-9e4a-5360198c8c5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:52.806626 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:52.820824 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:52.837222 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-4a1001a7-ba73-4c48-9e4a-5360198c8c5f None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:20:52.839361 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-4a1001a7-ba73-4c48-9e4a-5360198c8c5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:20:52.897559 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40843ef3-6434-455b-9d90-4b7d38284339 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:52.953064 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 250/503] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:20:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 31 12:20:52.962634 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cd60f1-84c8-4ab1-a0df-fb2aeaa7c39a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:52.963660 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cd60f1-84c8-4ab1-a0df-fb2aeaa7c39a None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:52.963775 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cd60f1-84c8-4ab1-a0df-fb2aeaa7c39a None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:52.963984 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cd60f1-84c8-4ab1-a0df-fb2aeaa7c39a None admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:52.964685 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cd60f1-84c8-4ab1-a0df-fb2aeaa7c39a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:52.964986 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cd60f1-84c8-4ab1-a0df-fb2aeaa7c39a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:52.966403 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cd60f1-84c8-4ab1-a0df-fb2aeaa7c39a None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:52.970670 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1001a7-ba73-4c48-9e4a-5360198c8c5f None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:52.981885 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 254/504] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Oct 31 12:20:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:52.991202 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cd60f1-84c8-4ab1-a0df-fb2aeaa7c39a None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:53.115280 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1001a7-ba73-4c48-9e4a-5360198c8c5f None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:53.195970 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560f7053-a122-461c-9199-17514ee813e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:53.197951 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560f7053-a122-461c-9199-17514ee813e9 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:53.199578 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560f7053-a122-461c-9199-17514ee813e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:53.200058 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560f7053-a122-461c-9199-17514ee813e9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:53.206288 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:53.222477 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560f7053-a122-461c-9199-17514ee813e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 31 12:20:53.228874 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-560f7053-a122-461c-9199-17514ee813e9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 31 12:20:53.228874 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:53.228874 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:53.228874 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:53.228874 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:53.228874 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:53.228874 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:53.228874 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:53.228874 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:53.228874 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:53.228874 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:53.228874 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:53.228874 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 31 12:20:53.228874 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 31 12:20:53.228874 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 31 12:20:53.228874 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 31 12:20:53.229797 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 31 12:20:53.229797 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 31 12:20:53.229797 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 31 12:20:53.229797 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 31 12:20:53.229797 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:53.229797 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:53.229797 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:53.229797 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:53.229797 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:53.229797 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:53.229797 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:53.229797 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:53.229797 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:53.229797 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:53.229797 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:53.229797 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:53.230667 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:53.230667 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:53.230667 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:53.230667 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:53.230667 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 31 12:20:53.230667 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 31 12:20:53.230667 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 31 12:20:53.230667 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 31 12:20:53.230667 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 31 12:20:53.230667 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 31 12:20:53.230667 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 31 12:20:53.230667 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:20:53.233739 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 251/505] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Oct 31 12:20:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 31 12:20:53.238069 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cd60f1-84c8-4ab1-a0df-fb2aeaa7c39a None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:53.239156 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:53.244043 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560f7053-a122-461c-9199-17514ee813e9 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:53.277108 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c727b0-99db-4c0b-b6d1-4e146036194a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:53.278098 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c727b0-99db-4c0b-b6d1-4e146036194a None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:53.278665 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c727b0-99db-4c0b-b6d1-4e146036194a None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:53.279072 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c727b0-99db-4c0b-b6d1-4e146036194a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:53.279995 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c727b0-99db-4c0b-b6d1-4e146036194a None admin] RBAC: Adding query filter params (role.id=9674a2c5acc74dfeb93cfd67dfd890dd, scope.project.id=2c32444cef584bce966b38203ce128b2, user.id=b2254a5786c249b4aa79244f7198ae65) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:53.280456 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c727b0-99db-4c0b-b6d1-4e146036194a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:53.282208 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c727b0-99db-4c0b-b6d1-4e146036194a None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:53.288936 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 255/506] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Oct 31 12:20:52 2022] GET /identity/v3/role_assignments?user.id=b2254a5786c249b4aa79244f7198ae65&scope.project.id=2c32444cef584bce966b38203ce128b2&role.id=9674a2c5acc74dfeb93cfd67dfd890dd => generated 608 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 31 12:20:53.290148 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c727b0-99db-4c0b-b6d1-4e146036194a None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:53.435600 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560f7053-a122-461c-9199-17514ee813e9 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:53.448416 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c727b0-99db-4c0b-b6d1-4e146036194a None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:53.489093 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460c4aad-2b3b-4247-bf4b-488ff6a22934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:53.490111 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460c4aad-2b3b-4247-bf4b-488ff6a22934 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:53.490585 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460c4aad-2b3b-4247-bf4b-488ff6a22934 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:53.491048 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460c4aad-2b3b-4247-bf4b-488ff6a22934 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:53.491867 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460c4aad-2b3b-4247-bf4b-488ff6a22934 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:53.493748 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460c4aad-2b3b-4247-bf4b-488ff6a22934 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:53.506944 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-460c4aad-2b3b-4247-bf4b-488ff6a22934 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 31 12:20:53.506944 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:53.506944 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 31 12:20:53.506944 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 31 12:20:53.506944 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 31 12:20:53.506944 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 31 12:20:53.506944 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 31 12:20:53.506944 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 31 12:20:53.506944 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 31 12:20:53.506944 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:20:53.506944 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 31 12:20:53.506944 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:20:53.506944 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:53.506944 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:53.506944 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:53.506944 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:53.506944 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:53.508239 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:53.508239 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:53.508239 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:53.508239 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:53.508239 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:53.508239 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:53.508239 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 31 12:20:53.508239 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 31 12:20:53.508239 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 31 12:20:53.508239 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 31 12:20:53.508239 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:53.508239 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:53.508239 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:53.508239 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:53.508239 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:53.508239 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:53.509186 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:53.509186 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:53.509186 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:53.509186 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:53.509186 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:53.509186 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:53.509186 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:53.509186 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:53.509186 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:53.509186 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:53.509186 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 31 12:20:53.509186 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 31 12:20:53.509186 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 31 12:20:53.509186 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:20:53.509186 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 256/507] 10.222.0.15 () {56 vars in 1129 bytes} [Mon Oct 31 12:20:53 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:20:53.518978 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460c4aad-2b3b-4247-bf4b-488ff6a22934 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:53.523187 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766774a5-6e2a-47ca-a5d2-2c3abf9f10f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:53.523187 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766774a5-6e2a-47ca-a5d2-2c3abf9f10f3 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:53.523187 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766774a5-6e2a-47ca-a5d2-2c3abf9f10f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:53.523187 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766774a5-6e2a-47ca-a5d2-2c3abf9f10f3 None admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:53.523850 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766774a5-6e2a-47ca-a5d2-2c3abf9f10f3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:53.523850 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766774a5-6e2a-47ca-a5d2-2c3abf9f10f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:53.523850 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766774a5-6e2a-47ca-a5d2-2c3abf9f10f3 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:53.538364 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 252/508] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Oct 31 12:20:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:53.545287 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766774a5-6e2a-47ca-a5d2-2c3abf9f10f3 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:53.574174 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:53.625205 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:53.656827 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460c4aad-2b3b-4247-bf4b-488ff6a22934 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:53.686524 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766774a5-6e2a-47ca-a5d2-2c3abf9f10f3 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:53.693799 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6584d9-5b3a-4022-aab2-32963091a892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:53.694589 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6584d9-5b3a-4022-aab2-32963091a892 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:53.695005 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6584d9-5b3a-4022-aab2-32963091a892 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:53.695464 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6584d9-5b3a-4022-aab2-32963091a892 None admin] PATH_INFO: `/v3/services` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:53.696230 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6584d9-5b3a-4022-aab2-32963091a892 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:53.696669 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6584d9-5b3a-4022-aab2-32963091a892 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:53.698681 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6584d9-5b3a-4022-aab2-32963091a892 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:53.703648 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 257/509] 10.222.0.15 () {56 vars in 1124 bytes} [Mon Oct 31 12:20:53 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:53.704923 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0eed67-a813-4968-96f9-4597dea7302f None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:53.705204 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0eed67-a813-4968-96f9-4597dea7302f None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:53.705470 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0eed67-a813-4968-96f9-4597dea7302f None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:53.710681 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 258/510] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:20:53 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:20:53.710681 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4c72c2-ecd6-4536-97c8-b2e418dc6144 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:53.710681 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4c72c2-ecd6-4536-97c8-b2e418dc6144 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:53.710681 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4c72c2-ecd6-4536-97c8-b2e418dc6144 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:53.710681 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 259/511] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:20:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:20:53.711086 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4c72c2-ecd6-4536-97c8-b2e418dc6144 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:53.733289 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5ce0b4-cd21-4932-870e-10f20229cceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:53.733919 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5ce0b4-cd21-4932-870e-10f20229cceb None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:53.734198 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5ce0b4-cd21-4932-870e-10f20229cceb None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:53.734467 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5ce0b4-cd21-4932-870e-10f20229cceb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:53.747032 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5ce0b4-cd21-4932-870e-10f20229cceb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:53.748437 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5ce0b4-cd21-4932-870e-10f20229cceb None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:53.765665 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-3c5ce0b4-cd21-4932-870e-10f20229cceb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:20:53.765665 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:53.765665 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:53.765665 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:53.765665 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:53.765665 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:53.765665 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:53.765665 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:53.765665 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:53.765665 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:53.765665 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:53.765665 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:53.765665 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 31 12:20:53.765665 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 31 12:20:53.765665 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 31 12:20:53.765665 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 31 12:20:53.766726 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:53.766726 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:53.766726 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:53.766726 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:53.766726 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:53.766726 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:53.766726 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:53.766726 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:53.766726 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:53.766726 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:53.766726 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:53.766726 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:53.766726 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:53.766726 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:53.766726 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:53.766726 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:53.766726 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 31 12:20:53.768103 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 31 12:20:53.768103 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:20:53.768103 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:20:53.768103 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 253/512] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Oct 31 12:20:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:20:53.768816 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17359c5e-92e7-427d-8ade-6007da71c2a5 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:53.769096 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17359c5e-92e7-427d-8ade-6007da71c2a5 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:53.769367 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17359c5e-92e7-427d-8ade-6007da71c2a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:53.791950 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-17359c5e-92e7-427d-8ade-6007da71c2a5 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:20:53.793722 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-17359c5e-92e7-427d-8ade-6007da71c2a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:20:53.824121 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:53.832122 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4c72c2-ecd6-4536-97c8-b2e418dc6144 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:53.870486 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7620f3-19fd-428d-b872-38a80b017a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:53.871106 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7620f3-19fd-428d-b872-38a80b017a1f None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:53.871468 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7620f3-19fd-428d-b872-38a80b017a1f None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:53.871570 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 254/513] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:20:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 31 12:20:53.871751 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7620f3-19fd-428d-b872-38a80b017a1f None admin] PATH_INFO: `/v3/services` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:53.872380 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7620f3-19fd-428d-b872-38a80b017a1f None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:53.872691 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7620f3-19fd-428d-b872-38a80b017a1f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:53.872783 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276b5607-7937-415c-a26f-4f5b12624c70 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:53.873065 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276b5607-7937-415c-a26f-4f5b12624c70 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:53.873344 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276b5607-7937-415c-a26f-4f5b12624c70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:53.874044 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7620f3-19fd-428d-b872-38a80b017a1f None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:53.890067 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 260/514] 10.222.0.15 () {56 vars in 1124 bytes} [Mon Oct 31 12:20:53 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:53.891016 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7620f3-19fd-428d-b872-38a80b017a1f None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:53.902260 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-276b5607-7937-415c-a26f-4f5b12624c70 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:20:53.903577 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-276b5607-7937-415c-a26f-4f5b12624c70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:20:53.964715 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:54.032019 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 255/515] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:20:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 31 12:20:54.084561 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7620f3-19fd-428d-b872-38a80b017a1f None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:54.122594 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6c8471-b7ee-4787-abad-0e22a4a00321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:54.123225 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6c8471-b7ee-4787-abad-0e22a4a00321 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:54.123503 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6c8471-b7ee-4787-abad-0e22a4a00321 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:54.123767 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6c8471-b7ee-4787-abad-0e22a4a00321 None admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:54.124402 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6c8471-b7ee-4787-abad-0e22a4a00321 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:54.124695 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6c8471-b7ee-4787-abad-0e22a4a00321 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:54.126047 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6c8471-b7ee-4787-abad-0e22a4a00321 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:54.139560 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 261/516] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Oct 31 12:20:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:54.145223 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276b5607-7937-415c-a26f-4f5b12624c70 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:54.212974 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6c8471-b7ee-4787-abad-0e22a4a00321 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:54.243502 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:54.311777 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276b5607-7937-415c-a26f-4f5b12624c70 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:54.342133 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6c8471-b7ee-4787-abad-0e22a4a00321 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:54.380216 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f988d43-c79b-46f8-8e98-f034143eeca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:54.381157 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f988d43-c79b-46f8-8e98-f034143eeca8 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:54.381807 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f988d43-c79b-46f8-8e98-f034143eeca8 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:54.382084 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f988d43-c79b-46f8-8e98-f034143eeca8 None admin] PATH_INFO: `/v3/services/network` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:54.382674 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f988d43-c79b-46f8-8e98-f034143eeca8 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:54.384363 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f988d43-c79b-46f8-8e98-f034143eeca8 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:54.395779 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-6f988d43-c79b-46f8-8e98-f034143eeca8 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 31 12:20:54.395779 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:54.395779 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 31 12:20:54.395779 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 31 12:20:54.395779 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 31 12:20:54.395779 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 31 12:20:54.395779 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 31 12:20:54.395779 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 31 12:20:54.395779 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 31 12:20:54.395779 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:20:54.395779 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 31 12:20:54.395779 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:20:54.395779 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:54.395779 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:54.395779 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:54.395779 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:54.395779 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:54.396812 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:54.396812 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:54.396812 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:54.396812 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:54.396812 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:54.396812 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:54.396812 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 31 12:20:54.396812 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 31 12:20:54.396812 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 31 12:20:54.396812 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 31 12:20:54.396812 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:54.396812 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:54.396812 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:54.396812 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:54.396812 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:54.396812 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:54.397700 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:54.397700 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:54.397700 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:54.397700 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:54.397700 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:54.397700 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:54.397700 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:54.397700 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:54.397700 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:54.397700 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:54.397700 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 31 12:20:54.397700 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 31 12:20:54.397700 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 31 12:20:54.397700 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:20:54.398406 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b261e17e-c5b0-4d03-a82a-ddc0da1609d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:54.398406 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b261e17e-c5b0-4d03-a82a-ddc0da1609d9 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:54.398406 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b261e17e-c5b0-4d03-a82a-ddc0da1609d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:54.398406 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b261e17e-c5b0-4d03-a82a-ddc0da1609d9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:54.398674 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 262/517] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Oct 31 12:20:54 2022] GET /identity/v3/services/network => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 31 12:20:54.410609 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b261e17e-c5b0-4d03-a82a-ddc0da1609d9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:54.412086 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b261e17e-c5b0-4d03-a82a-ddc0da1609d9 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:54.413938 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f988d43-c79b-46f8-8e98-f034143eeca8 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:54.426738 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-b261e17e-c5b0-4d03-a82a-ddc0da1609d9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 31 12:20:54.426738 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:54.426738 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:54.426738 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:54.426738 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:54.426738 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:54.426738 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:54.426738 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:54.426738 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:54.426738 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:54.426738 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:54.426738 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:54.426738 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 31 12:20:54.426738 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 31 12:20:54.426738 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 31 12:20:54.426738 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 31 12:20:54.428882 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:54.428882 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:54.428882 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 31 12:20:54.428882 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 31 12:20:54.428882 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 31 12:20:54.428882 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 31 12:20:54.428882 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:54.428882 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:54.428882 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:54.428882 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:54.428882 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:54.428882 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:54.428882 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:54.428882 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:54.428882 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:54.428882 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:54.428882 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:54.428882 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:54.431042 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:54.431042 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:54.431042 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 31 12:20:54.431042 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 31 12:20:54.431042 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 31 12:20:54.431042 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 31 12:20:54.431042 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 31 12:20:54.431042 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 31 12:20:54.431042 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 31 12:20:54.431042 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:20:54.435186 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 256/518] 10.222.0.15 () {56 vars in 1093 bytes} [Mon Oct 31 12:20:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:20:54.435186 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b261e17e-c5b0-4d03-a82a-ddc0da1609d9 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:54.466958 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:54.537774 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:54.550820 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b261e17e-c5b0-4d03-a82a-ddc0da1609d9 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:54.561149 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f988d43-c79b-46f8-8e98-f034143eeca8 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:54.592127 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df87898-e49d-465f-b93d-6aa3df6111ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:54.592661 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df87898-e49d-465f-b93d-6aa3df6111ba None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:54.592937 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df87898-e49d-465f-b93d-6aa3df6111ba None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:54.593263 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df87898-e49d-465f-b93d-6aa3df6111ba None admin] PATH_INFO: `/v3/users` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:54.594127 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df87898-e49d-465f-b93d-6aa3df6111ba None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:54.594381 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df87898-e49d-465f-b93d-6aa3df6111ba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:54.595927 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df87898-e49d-465f-b93d-6aa3df6111ba None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:54.600760 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a787815-8f34-47c7-bb73-2ce0c7b59337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:54.601622 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a787815-8f34-47c7-bb73-2ce0c7b59337 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:54.602093 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a787815-8f34-47c7-bb73-2ce0c7b59337 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:54.602552 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a787815-8f34-47c7-bb73-2ce0c7b59337 None admin] PATH_INFO: `/v3/services` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:54.603419 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a787815-8f34-47c7-bb73-2ce0c7b59337 None admin] RBAC: Adding query filter params (name=network) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:54.604025 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a787815-8f34-47c7-bb73-2ce0c7b59337 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:54.605555 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a787815-8f34-47c7-bb73-2ce0c7b59337 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:54.616346 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 263/519] 10.222.0.15 () {56 vars in 1112 bytes} [Mon Oct 31 12:20:54 2022] GET /identity/v3/services?name=network => generated 124 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 31 12:20:54.616346 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a787815-8f34-47c7-bb73-2ce0c7b59337 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:54.625261 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 257/520] 10.222.0.15 () {56 vars in 1133 bytes} [Mon Oct 31 12:20:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:54.625261 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df87898-e49d-465f-b93d-6aa3df6111ba None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:54.719679 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:54.739265 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a787815-8f34-47c7-bb73-2ce0c7b59337 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:54.740278 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df87898-e49d-465f-b93d-6aa3df6111ba None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:54.777889 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fa22a2-02d6-4a62-8dd5-f075a01d36ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:54.778820 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fa22a2-02d6-4a62-8dd5-f075a01d36ab None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:54.779103 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fa22a2-02d6-4a62-8dd5-f075a01d36ab None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:54.779404 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fa22a2-02d6-4a62-8dd5-f075a01d36ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:54.785327 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41b222b-e09c-4186-b489-ea7b8f1be127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:54.786206 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41b222b-e09c-4186-b489-ea7b8f1be127 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:54.786694 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41b222b-e09c-4186-b489-ea7b8f1be127 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:54.787157 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41b222b-e09c-4186-b489-ea7b8f1be127 None admin] PATH_INFO: `/v3/services` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:54.788453 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41b222b-e09c-4186-b489-ea7b8f1be127 None admin] RBAC: Adding query filter params (type=network) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:54.788952 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41b222b-e09c-4186-b489-ea7b8f1be127 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:54.791316 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41b222b-e09c-4186-b489-ea7b8f1be127 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:54.791912 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fa22a2-02d6-4a62-8dd5-f075a01d36ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:54.793522 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fa22a2-02d6-4a62-8dd5-f075a01d36ab None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:54.803837 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 258/521] 10.222.0.15 () {56 vars in 1112 bytes} [Mon Oct 31 12:20:54 2022] GET /identity/v3/services?type=network => generated 124 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:54.806478 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41b222b-e09c-4186-b489-ea7b8f1be127 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:54.823610 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:54.826089 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-48fa22a2-02d6-4a62-8dd5-f075a01d36ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:20:54.826089 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:54.826089 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:54.826089 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:54.826089 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:54.826089 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:54.826089 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:54.826089 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:54.826089 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:54.826089 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:54.826089 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:54.826089 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:54.826089 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 31 12:20:54.826089 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 31 12:20:54.826089 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 31 12:20:54.826089 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 31 12:20:54.828345 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:54.828345 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:54.828345 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:54.828345 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:54.828345 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:54.828345 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:54.828345 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:54.828345 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:54.828345 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:54.828345 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:54.828345 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:54.828345 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:54.828345 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:54.828345 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:54.828345 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:54.828345 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:54.828345 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 31 12:20:54.829232 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 31 12:20:54.829232 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:20:54.829232 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:20:54.829232 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 264/522] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Oct 31 12:20:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:20:54.837196 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fa22a2-02d6-4a62-8dd5-f075a01d36ab None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:54.998181 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41b222b-e09c-4186-b489-ea7b8f1be127 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:55.037188 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fa22a2-02d6-4a62-8dd5-f075a01d36ab None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:55.063999 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c672035d-7cf3-4cc7-835a-6d2a21fe9354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:55.064512 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c672035d-7cf3-4cc7-835a-6d2a21fe9354 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:55.064800 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c672035d-7cf3-4cc7-835a-6d2a21fe9354 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:55.065072 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c672035d-7cf3-4cc7-835a-6d2a21fe9354 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:55.080230 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c672035d-7cf3-4cc7-835a-6d2a21fe9354 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:55.081643 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfb2252-850b-4473-9c6d-e196aca2cd37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:55.081796 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c672035d-7cf3-4cc7-835a-6d2a21fe9354 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:55.082615 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfb2252-850b-4473-9c6d-e196aca2cd37 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:55.083137 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfb2252-850b-4473-9c6d-e196aca2cd37 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:55.083737 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfb2252-850b-4473-9c6d-e196aca2cd37 None admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:55.085372 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfb2252-850b-4473-9c6d-e196aca2cd37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:55.085881 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfb2252-850b-4473-9c6d-e196aca2cd37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:55.087447 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfb2252-850b-4473-9c6d-e196aca2cd37 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:55.100699 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-c672035d-7cf3-4cc7-835a-6d2a21fe9354 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:20:55.100699 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:55.100699 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:55.100699 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:55.100699 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:55.100699 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:55.100699 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:55.100699 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:55.100699 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:55.100699 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:55.100699 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:55.100699 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:55.100699 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 31 12:20:55.100699 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 31 12:20:55.100699 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 31 12:20:55.100699 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 31 12:20:55.101674 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:55.101674 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:55.101674 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:55.101674 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:55.101674 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:55.101674 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:55.101674 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:55.101674 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:55.101674 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:55.101674 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:55.101674 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:55.101674 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:55.101674 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:55.101674 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:55.101674 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:55.101674 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:55.101674 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 31 12:20:55.102552 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 31 12:20:55.102552 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:20:55.102552 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:20:55.102552 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 259/523] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Oct 31 12:20:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:20:55.108789 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c672035d-7cf3-4cc7-835a-6d2a21fe9354 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:55.109073 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 265/524] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Oct 31 12:20:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:55.114199 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfb2252-850b-4473-9c6d-e196aca2cd37 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:55.115470 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:55.223528 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:55.292014 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfb2252-850b-4473-9c6d-e196aca2cd37 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:55.329020 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d07b352-aaff-44fa-99be-d7e0fee6069a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:55.329780 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d07b352-aaff-44fa-99be-d7e0fee6069a None admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:55.330158 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d07b352-aaff-44fa-99be-d7e0fee6069a None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:55.330526 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d07b352-aaff-44fa-99be-d7e0fee6069a None admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:55.331349 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d07b352-aaff-44fa-99be-d7e0fee6069a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:55.332948 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d07b352-aaff-44fa-99be-d7e0fee6069a None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:55.344325 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c672035d-7cf3-4cc7-835a-6d2a21fe9354 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:55.362239 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 266/525] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Oct 31 12:20:55 2022] POST /identity/v3/users => generated 254 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:20:55.404648 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eea48d7-9704-4221-a24c-4a6e0afd84b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:55.405424 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eea48d7-9704-4221-a24c-4a6e0afd84b4 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:55.405855 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eea48d7-9704-4221-a24c-4a6e0afd84b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:55.406222 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eea48d7-9704-4221-a24c-4a6e0afd84b4 None admin] PATH_INFO: `/v3/domains` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:55.406981 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eea48d7-9704-4221-a24c-4a6e0afd84b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:55.407465 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eea48d7-9704-4221-a24c-4a6e0afd84b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:55.408946 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eea48d7-9704-4221-a24c-4a6e0afd84b4 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:55.424574 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 260/526] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Oct 31 12:20:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:55.432991 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d07b352-aaff-44fa-99be-d7e0fee6069a None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:55.580766 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d07b352-aaff-44fa-99be-d7e0fee6069a None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:55.610249 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:55.633750 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e0d1d9-5936-4f08-864d-193f90a54e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:55.634431 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e0d1d9-5936-4f08-864d-193f90a54e39 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:55.635281 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e0d1d9-5936-4f08-864d-193f90a54e39 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:55.636137 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e0d1d9-5936-4f08-864d-193f90a54e39 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:55.642917 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e0d1d9-5936-4f08-864d-193f90a54e39 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:55.644651 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e0d1d9-5936-4f08-864d-193f90a54e39 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:55.657680 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-d9e0d1d9-5936-4f08-864d-193f90a54e39 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 31 12:20:55.657680 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:55.657680 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:55.657680 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:55.657680 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:55.657680 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:55.657680 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:55.657680 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:55.657680 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:55.657680 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:55.657680 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:55.657680 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:55.657680 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 31 12:20:55.657680 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 31 12:20:55.657680 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 31 12:20:55.658585 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 31 12:20:55.658585 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:55.658585 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:55.658585 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:55.658585 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:55.658585 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:55.658585 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:55.658585 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:55.658585 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:55.658585 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:55.658585 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:55.658585 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:55.658585 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:55.658585 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:55.658585 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:55.658585 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:55.658585 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:55.659485 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 31 12:20:55.659485 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 31 12:20:55.659485 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 31 12:20:55.659485 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 31 12:20:55.659485 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 31 12:20:55.659485 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 31 12:20:55.659485 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 31 12:20:55.659485 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:20:55.659485 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 267/527] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Oct 31 12:20:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:20:55.664490 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eea48d7-9704-4221-a24c-4a6e0afd84b4 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:55.762935 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:55.811492 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eea48d7-9704-4221-a24c-4a6e0afd84b4 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:55.840501 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2612adb4-d500-405b-ab8b-307d44202287 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:55.840992 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2612adb4-d500-405b-ab8b-307d44202287 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:55.840992 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2612adb4-d500-405b-ab8b-307d44202287 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:55.842547 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 268/528] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:20:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:20:55.848009 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d415723-dfda-4679-a8a8-6df53ff724e2 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:55.848275 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d415723-dfda-4679-a8a8-6df53ff724e2 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:55.848542 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d415723-dfda-4679-a8a8-6df53ff724e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:55.862654 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97bcd76-5d7d-41d7-98a1-60f14b051ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:55.863693 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97bcd76-5d7d-41d7-98a1-60f14b051ebb None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:55.864185 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97bcd76-5d7d-41d7-98a1-60f14b051ebb None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:55.864681 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97bcd76-5d7d-41d7-98a1-60f14b051ebb None admin] PATH_INFO: `/v3/projects` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:55.865551 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97bcd76-5d7d-41d7-98a1-60f14b051ebb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:55.866043 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97bcd76-5d7d-41d7-98a1-60f14b051ebb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:55.867687 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97bcd76-5d7d-41d7-98a1-60f14b051ebb None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:55.867973 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-7d415723-dfda-4679-a8a8-6df53ff724e2 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:20:55.869223 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-7d415723-dfda-4679-a8a8-6df53ff724e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:20:55.889006 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 261/529] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Oct 31 12:20:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:55.904964 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97bcd76-5d7d-41d7-98a1-60f14b051ebb None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:55.953447 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 269/530] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:20:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 31 12:20:56.081711 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97bcd76-5d7d-41d7-98a1-60f14b051ebb None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:56.113724 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:56.131872 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b39d949-f441-4a3b-b1e0-084e7ee04c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:56.132532 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b39d949-f441-4a3b-b1e0-084e7ee04c11 None admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:56.132806 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b39d949-f441-4a3b-b1e0-084e7ee04c11 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:56.133075 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b39d949-f441-4a3b-b1e0-084e7ee04c11 None admin] PATH_INFO: `/v3/projects/2c32444cef584bce966b38203ce128b2/users/99889b1dab0b471d9c77887cffefc652/roles/176c07b4fe3b4537bdfeb4be21b379c5` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:56.137899 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b39d949-f441-4a3b-b1e0-084e7ee04c11 None admin] RBAC: Authorizing `identity:create_grant(project_id=2c32444cef584bce966b38203ce128b2, user_id=99889b1dab0b471d9c77887cffefc652, role_id=176c07b4fe3b4537bdfeb4be21b379c5)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:56.139346 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b39d949-f441-4a3b-b1e0-084e7ee04c11 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:56.162748 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 262/531] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Oct 31 12:20:55 2022] PUT /identity/v3/projects/2c32444cef584bce966b38203ce128b2/users/99889b1dab0b471d9c77887cffefc652/roles/176c07b4fe3b4537bdfeb4be21b379c5 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 31 12:20:56.280153 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d415723-dfda-4679-a8a8-6df53ff724e2 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:56.403040 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d415723-dfda-4679-a8a8-6df53ff724e2 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:56.419014 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:56.460890 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab5a2d0-01a6-45d8-914b-a877e59c9bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:56.461905 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab5a2d0-01a6-45d8-914b-a877e59c9bf9 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:56.462404 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab5a2d0-01a6-45d8-914b-a877e59c9bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:56.463520 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab5a2d0-01a6-45d8-914b-a877e59c9bf9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:56.477907 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab5a2d0-01a6-45d8-914b-a877e59c9bf9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 31 12:20:56.483627 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-7ab5a2d0-01a6-45d8-914b-a877e59c9bf9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 31 12:20:56.483627 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:56.483627 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:56.483627 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:56.483627 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:56.483627 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:56.483627 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:56.483627 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:56.483627 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:56.483627 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:56.483627 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:56.483627 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:56.483627 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 31 12:20:56.483627 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 31 12:20:56.483627 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 31 12:20:56.483627 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 31 12:20:56.484527 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 31 12:20:56.484527 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 31 12:20:56.484527 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 31 12:20:56.484527 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 31 12:20:56.484527 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:56.484527 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:56.484527 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:56.484527 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:56.484527 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:56.484527 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:56.484527 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:56.484527 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:56.484527 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:56.484527 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:56.484527 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:56.484527 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:56.485350 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:56.485350 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:56.485350 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:56.485350 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:56.485350 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 31 12:20:56.485350 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 31 12:20:56.485350 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 31 12:20:56.485350 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 31 12:20:56.485350 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 31 12:20:56.485350 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 31 12:20:56.485350 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 31 12:20:56.485350 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:20:56.487970 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 270/532] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Oct 31 12:20:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:20:56.493396 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b39d949-f441-4a3b-b1e0-084e7ee04c11 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:56.574046 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4498b7d-b307-4760-894c-4f3762423cb9 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:56.574392 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4498b7d-b307-4760-894c-4f3762423cb9 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:56.574602 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4498b7d-b307-4760-894c-4f3762423cb9 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:56.576245 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 271/533] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:20:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:20:56.581214 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6968c5-1f8f-4377-b250-f84e013d19df None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:56.581503 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6968c5-1f8f-4377-b250-f84e013d19df None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:56.581775 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6968c5-1f8f-4377-b250-f84e013d19df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:56.600912 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-5c6968c5-1f8f-4377-b250-f84e013d19df None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:20:56.602096 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-5c6968c5-1f8f-4377-b250-f84e013d19df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:20:56.637206 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b39d949-f441-4a3b-b1e0-084e7ee04c11 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:56.677473 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 272/534] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:20:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 31 12:20:56.681448 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd71630-6117-4831-9b3f-fc1448fe6794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:56.682294 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd71630-6117-4831-9b3f-fc1448fe6794 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:56.682573 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd71630-6117-4831-9b3f-fc1448fe6794 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:56.682866 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd71630-6117-4831-9b3f-fc1448fe6794 None admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:56.683594 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd71630-6117-4831-9b3f-fc1448fe6794 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:56.683891 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd71630-6117-4831-9b3f-fc1448fe6794 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:56.685412 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd71630-6117-4831-9b3f-fc1448fe6794 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:56.705209 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 263/535] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Oct 31 12:20:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:56.706223 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6968c5-1f8f-4377-b250-f84e013d19df None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:56.829147 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:56.835264 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6968c5-1f8f-4377-b250-f84e013d19df None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:56.872553 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872e3fc6-c68c-486a-9d0d-d3555803dc3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:56.873316 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872e3fc6-c68c-486a-9d0d-d3555803dc3a None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:56.873497 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872e3fc6-c68c-486a-9d0d-d3555803dc3a None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:56.873704 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872e3fc6-c68c-486a-9d0d-d3555803dc3a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:56.884696 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872e3fc6-c68c-486a-9d0d-d3555803dc3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:56.886743 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872e3fc6-c68c-486a-9d0d-d3555803dc3a None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:56.902296 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-872e3fc6-c68c-486a-9d0d-d3555803dc3a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:20:56.902296 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:56.902296 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:56.902296 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:56.902296 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:56.902296 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:56.902296 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:56.902296 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:56.902296 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:56.902296 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:56.902296 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:56.902296 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:56.902296 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 31 12:20:56.902296 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 31 12:20:56.902296 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 31 12:20:56.902296 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 31 12:20:56.904223 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:56.904223 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:56.904223 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:56.904223 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:56.904223 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:56.904223 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:56.904223 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:56.904223 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:56.904223 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:56.904223 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:56.904223 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:56.904223 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:56.904223 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:56.904223 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:56.904223 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:56.904223 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:56.904223 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 31 12:20:56.906112 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 31 12:20:56.906112 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:20:56.906112 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:20:56.906112 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 273/536] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Oct 31 12:20:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:20:56.909015 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd71630-6117-4831-9b3f-fc1448fe6794 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:56.983411 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:57.051423 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd71630-6117-4831-9b3f-fc1448fe6794 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:57.101750 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a726d051-dff2-4b52-bea4-9c850a15f621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:57.102368 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a726d051-dff2-4b52-bea4-9c850a15f621 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:57.102709 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a726d051-dff2-4b52-bea4-9c850a15f621 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:57.102984 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a726d051-dff2-4b52-bea4-9c850a15f621 None admin] PATH_INFO: `/v3/domains` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:57.103722 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a726d051-dff2-4b52-bea4-9c850a15f621 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:57.104339 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a726d051-dff2-4b52-bea4-9c850a15f621 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:57.105368 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a726d051-dff2-4b52-bea4-9c850a15f621 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:57.115496 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872e3fc6-c68c-486a-9d0d-d3555803dc3a None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:57.126261 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 264/537] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Oct 31 12:20:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:57.131298 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a726d051-dff2-4b52-bea4-9c850a15f621 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:57.271127 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a726d051-dff2-4b52-bea4-9c850a15f621 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:57.299608 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:57.331189 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491b5179-2c6c-492e-89c6-c76377513b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:57.331430 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872e3fc6-c68c-486a-9d0d-d3555803dc3a None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:57.331517 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491b5179-2c6c-492e-89c6-c76377513b60 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:57.331517 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491b5179-2c6c-492e-89c6-c76377513b60 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:57.331517 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491b5179-2c6c-492e-89c6-c76377513b60 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:57.348611 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491b5179-2c6c-492e-89c6-c76377513b60 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:57.350065 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491b5179-2c6c-492e-89c6-c76377513b60 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:57.371917 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-491b5179-2c6c-492e-89c6-c76377513b60 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 31 12:20:57.371917 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:57.371917 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:57.371917 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:57.371917 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:57.371917 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:57.371917 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:57.371917 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:57.371917 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:57.371917 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:57.371917 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:57.371917 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:57.371917 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 31 12:20:57.371917 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 31 12:20:57.371917 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 31 12:20:57.371917 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 31 12:20:57.372915 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:57.372915 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:57.372915 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 31 12:20:57.372915 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 31 12:20:57.372915 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 31 12:20:57.372915 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 31 12:20:57.372915 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:57.372915 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:57.372915 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:57.372915 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:57.372915 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:57.372915 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:57.372915 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:57.372915 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:57.372915 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:57.372915 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:57.372915 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:57.372915 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:57.373840 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:57.373840 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:57.373840 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 31 12:20:57.373840 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 31 12:20:57.373840 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 31 12:20:57.373840 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 31 12:20:57.373840 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 31 12:20:57.373840 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 31 12:20:57.373840 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 31 12:20:57.373840 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:20:57.373840 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 265/538] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Oct 31 12:20:57 2022] GET /identity/v3/users/swift => generated 83 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:20:57.378862 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491b5179-2c6c-492e-89c6-c76377513b60 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:57.395913 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bbb594-83a0-4d4d-ba87-3184257523ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:57.396539 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bbb594-83a0-4d4d-ba87-3184257523ee None admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:57.396815 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bbb594-83a0-4d4d-ba87-3184257523ee None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:57.397083 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bbb594-83a0-4d4d-ba87-3184257523ee None admin] PATH_INFO: `/v3/services` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:57.397748 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bbb594-83a0-4d4d-ba87-3184257523ee None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:57.399100 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bbb594-83a0-4d4d-ba87-3184257523ee None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:57.461121 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 274/539] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Oct 31 12:20:57 2022] POST /identity/v3/services => generated 251 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 31 12:20:57.564811 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab803b74-179e-4735-931b-06a518833b40 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:57.565186 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab803b74-179e-4735-931b-06a518833b40 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:57.565358 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab803b74-179e-4735-931b-06a518833b40 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:57.567040 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 275/540] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:20:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:20:57.584695 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3cfaae-e2c4-4ebc-84be-9a6178f8c2e1 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:57.585014 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3cfaae-e2c4-4ebc-84be-9a6178f8c2e1 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:57.585228 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3cfaae-e2c4-4ebc-84be-9a6178f8c2e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:57.631323 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491b5179-2c6c-492e-89c6-c76377513b60 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:57.652547 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-ca3cfaae-e2c4-4ebc-84be-9a6178f8c2e1 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:20:57.653758 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-ca3cfaae-e2c4-4ebc-84be-9a6178f8c2e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:20:57.656152 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:57.719883 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d09cfe-93ae-4763-b331-9588740d8ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:57.720520 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d09cfe-93ae-4763-b331-9588740d8ccc None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:57.720757 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d09cfe-93ae-4763-b331-9588740d8ccc None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:57.721020 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d09cfe-93ae-4763-b331-9588740d8ccc None admin] PATH_INFO: `/v3/users` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:57.721737 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d09cfe-93ae-4763-b331-9588740d8ccc None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:57.726006 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d09cfe-93ae-4763-b331-9588740d8ccc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:57.727529 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d09cfe-93ae-4763-b331-9588740d8ccc None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:57.760353 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 266/541] 10.222.0.15 () {56 vars in 1135 bytes} [Mon Oct 31 12:20:57 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 379 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 31 12:20:57.764116 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d09cfe-93ae-4763-b331-9588740d8ccc None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:57.786163 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:57.856049 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 276/542] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:20:57 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 31 12:20:57.938988 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d09cfe-93ae-4763-b331-9588740d8ccc None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:57.984244 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8538fd8c-f33e-44ba-acab-2483dc1bcfaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:57.984934 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8538fd8c-f33e-44ba-acab-2483dc1bcfaa None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:57.985212 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8538fd8c-f33e-44ba-acab-2483dc1bcfaa None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:57.985482 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8538fd8c-f33e-44ba-acab-2483dc1bcfaa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:58.000730 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8538fd8c-f33e-44ba-acab-2483dc1bcfaa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:58.002202 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8538fd8c-f33e-44ba-acab-2483dc1bcfaa None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:58.022232 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-8538fd8c-f33e-44ba-acab-2483dc1bcfaa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:20:58.022232 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:58.022232 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:58.022232 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:58.022232 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:58.022232 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:58.022232 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:58.022232 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:58.022232 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:58.022232 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:58.022232 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:58.022232 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:58.022232 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 31 12:20:58.022232 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 31 12:20:58.022232 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 31 12:20:58.022232 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 31 12:20:58.023252 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:58.023252 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:58.023252 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:58.023252 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:58.023252 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:58.023252 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:58.023252 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:58.023252 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:58.023252 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:58.023252 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:58.023252 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:58.023252 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:58.023252 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:58.023252 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:58.023252 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:58.023252 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:58.023252 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 31 12:20:58.024748 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 31 12:20:58.024748 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:20:58.024748 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:20:58.024748 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 267/543] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Oct 31 12:20:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:20:58.029674 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3cfaae-e2c4-4ebc-84be-9a6178f8c2e1 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:58.141246 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3cfaae-e2c4-4ebc-84be-9a6178f8c2e1 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:58.180520 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f65a144-8e3e-4d8c-8a4c-57e664b5b52e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:58.181519 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f65a144-8e3e-4d8c-8a4c-57e664b5b52e None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:58.182045 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f65a144-8e3e-4d8c-8a4c-57e664b5b52e None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:58.182568 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f65a144-8e3e-4d8c-8a4c-57e664b5b52e None admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:58.183422 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f65a144-8e3e-4d8c-8a4c-57e664b5b52e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:58.183917 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f65a144-8e3e-4d8c-8a4c-57e664b5b52e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:58.185525 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f65a144-8e3e-4d8c-8a4c-57e664b5b52e None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:58.197882 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:58.198252 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 277/544] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Oct 31 12:20:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:58.204499 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8538fd8c-f33e-44ba-acab-2483dc1bcfaa None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:58.249342 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f65a144-8e3e-4d8c-8a4c-57e664b5b52e None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:58.334039 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:58.367065 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f65a144-8e3e-4d8c-8a4c-57e664b5b52e None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:58.396942 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8538fd8c-f33e-44ba-acab-2483dc1bcfaa None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:58.413254 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e747bd-5081-49bd-9f85-98d6f7180390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:58.413873 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e747bd-5081-49bd-9f85-98d6f7180390 None admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:58.414145 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e747bd-5081-49bd-9f85-98d6f7180390 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:58.414449 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e747bd-5081-49bd-9f85-98d6f7180390 None admin] PATH_INFO: `/v3/services` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:58.415065 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e747bd-5081-49bd-9f85-98d6f7180390 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:58.416756 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e747bd-5081-49bd-9f85-98d6f7180390 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:58.439265 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 278/545] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Oct 31 12:20:58 2022] POST /identity/v3/services => generated 240 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 31 12:20:58.440736 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f72c4e1-d4b6-40f8-8670-23325014e5b2 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:58.441018 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f72c4e1-d4b6-40f8-8670-23325014e5b2 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:58.441288 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f72c4e1-d4b6-40f8-8670-23325014e5b2 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:58.442714 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 279/546] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:20:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:20:58.450189 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0b822c-a24a-456f-aaa0-45f3f06453a0 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:58.450506 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0b822c-a24a-456f-aaa0-45f3f06453a0 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:58.450736 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0b822c-a24a-456f-aaa0-45f3f06453a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:58.485394 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-8d0b822c-a24a-456f-aaa0-45f3f06453a0 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:20:58.486655 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-8d0b822c-a24a-456f-aaa0-45f3f06453a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:20:58.516933 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff11c22a-3414-4169-8542-f461ca9d9979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:58.517959 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff11c22a-3414-4169-8542-f461ca9d9979 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:58.518240 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff11c22a-3414-4169-8542-f461ca9d9979 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:58.518510 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff11c22a-3414-4169-8542-f461ca9d9979 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:58.535777 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff11c22a-3414-4169-8542-f461ca9d9979 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:58.537253 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff11c22a-3414-4169-8542-f461ca9d9979 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:58.562855 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-ff11c22a-3414-4169-8542-f461ca9d9979 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 31 12:20:58.562855 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:58.562855 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:58.562855 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:58.562855 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:58.562855 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:58.562855 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:58.562855 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:58.562855 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:58.562855 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:58.562855 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:58.562855 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:58.562855 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 31 12:20:58.562855 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 31 12:20:58.562855 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 31 12:20:58.563850 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 31 12:20:58.563850 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:58.563850 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:58.563850 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:58.563850 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:58.563850 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:58.563850 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:58.563850 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:58.563850 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:58.563850 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:58.563850 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:58.563850 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:58.563850 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:58.563850 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:58.563850 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:58.563850 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:58.563850 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:58.564718 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 31 12:20:58.564718 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 31 12:20:58.564718 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 31 12:20:58.564718 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 31 12:20:58.564718 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 31 12:20:58.564718 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 31 12:20:58.564718 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 31 12:20:58.564718 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:20:58.568212 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 268/547] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Oct 31 12:20:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 364 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:20:58.574412 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff11c22a-3414-4169-8542-f461ca9d9979 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:58.589918 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:58.610593 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 280/548] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:20:58 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 31 12:20:58.710772 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff11c22a-3414-4169-8542-f461ca9d9979 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:58.741266 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:58.752035 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b0a4aa-7d83-4099-ab0c-2b387138957f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:58.752645 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b0a4aa-7d83-4099-ab0c-2b387138957f None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:58.752923 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b0a4aa-7d83-4099-ab0c-2b387138957f None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:58.753196 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b0a4aa-7d83-4099-ab0c-2b387138957f None admin] PATH_INFO: `/v3/projects` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:58.753881 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b0a4aa-7d83-4099-ab0c-2b387138957f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:58.754180 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b0a4aa-7d83-4099-ab0c-2b387138957f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:58.755632 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b0a4aa-7d83-4099-ab0c-2b387138957f None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:58.769234 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 269/549] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Oct 31 12:20:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:58.775012 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0b822c-a24a-456f-aaa0-45f3f06453a0 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:58.902877 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0b822c-a24a-456f-aaa0-45f3f06453a0 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:58.951351 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0539a24e-fbf8-43e2-bfce-80001c401351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:58.952394 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0539a24e-fbf8-43e2-bfce-80001c401351 None admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:58.952867 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0539a24e-fbf8-43e2-bfce-80001c401351 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:58.953330 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0539a24e-fbf8-43e2-bfce-80001c401351 None admin] PATH_INFO: `/v3/projects/2c32444cef584bce966b38203ce128b2/users/b2254a5786c249b4aa79244f7198ae65/roles/176c07b4fe3b4537bdfeb4be21b379c5` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:58.955459 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0539a24e-fbf8-43e2-bfce-80001c401351 None admin] RBAC: Authorizing `identity:create_grant(project_id=2c32444cef584bce966b38203ce128b2, user_id=b2254a5786c249b4aa79244f7198ae65, role_id=176c07b4fe3b4537bdfeb4be21b379c5)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:58.957110 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0539a24e-fbf8-43e2-bfce-80001c401351 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:58.969885 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b0a4aa-7d83-4099-ab0c-2b387138957f None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:58.977207 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 281/550] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Oct 31 12:20:58 2022] PUT /identity/v3/projects/2c32444cef584bce966b38203ce128b2/users/b2254a5786c249b4aa79244f7198ae65/roles/176c07b4fe3b4537bdfeb4be21b379c5 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:20:58.978646 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69f78dd-abdf-4cb3-b587-993ed77dc0d2 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:58.979033 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69f78dd-abdf-4cb3-b587-993ed77dc0d2 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:58.979325 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69f78dd-abdf-4cb3-b587-993ed77dc0d2 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:58.980952 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 282/551] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:20:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:20:58.993855 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5813c4da-522a-4087-af90-154458c08558 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:58.994831 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5813c4da-522a-4087-af90-154458c08558 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:58.995193 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5813c4da-522a-4087-af90-154458c08558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:59.040362 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-5813c4da-522a-4087-af90-154458c08558 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:20:59.041842 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-5813c4da-522a-4087-af90-154458c08558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:20:59.121700 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:59.168760 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b0a4aa-7d83-4099-ab0c-2b387138957f None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:59.182411 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 283/552] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:20:58 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 31 12:20:59.208541 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80f4151-4d8d-4171-9ae9-a8b770cdc38b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:59.209173 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80f4151-4d8d-4171-9ae9-a8b770cdc38b None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:59.209446 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80f4151-4d8d-4171-9ae9-a8b770cdc38b None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:59.209710 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80f4151-4d8d-4171-9ae9-a8b770cdc38b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:59.223120 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80f4151-4d8d-4171-9ae9-a8b770cdc38b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 31 12:20:59.228088 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-f80f4151-4d8d-4171-9ae9-a8b770cdc38b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 31 12:20:59.228088 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:59.228088 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:59.228088 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:59.228088 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:59.228088 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:59.228088 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:59.228088 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:59.228088 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:59.228088 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:59.228088 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:59.228088 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:59.228088 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 31 12:20:59.228088 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 31 12:20:59.228088 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 31 12:20:59.228088 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 31 12:20:59.229214 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 31 12:20:59.229214 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 31 12:20:59.229214 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 31 12:20:59.229214 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 31 12:20:59.229214 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:59.229214 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:59.229214 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:59.229214 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:59.229214 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:59.229214 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:59.229214 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:59.229214 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:59.229214 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:59.229214 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:59.229214 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:59.229214 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:59.230038 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:59.230038 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:59.230038 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:59.230038 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:59.230038 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 31 12:20:59.230038 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 31 12:20:59.230038 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 31 12:20:59.230038 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 31 12:20:59.230038 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 31 12:20:59.230038 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 31 12:20:59.230038 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 31 12:20:59.230038 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:20:59.230038 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 270/553] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Oct 31 12:20:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 31 12:20:59.237523 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5813c4da-522a-4087-af90-154458c08558 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:59.340005 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5813c4da-522a-4087-af90-154458c08558 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:59.374632 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d28f44-ddaf-40e3-b446-305a05a50681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:59.375577 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d28f44-ddaf-40e3-b446-305a05a50681 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:59.376032 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d28f44-ddaf-40e3-b446-305a05a50681 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:59.376476 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d28f44-ddaf-40e3-b446-305a05a50681 None admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:59.377300 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d28f44-ddaf-40e3-b446-305a05a50681 None admin] RBAC: Adding query filter params (name=service) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:59.377789 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d28f44-ddaf-40e3-b446-305a05a50681 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:59.379350 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d28f44-ddaf-40e3-b446-305a05a50681 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:59.388544 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 284/554] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Oct 31 12:20:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:59.394370 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80f4151-4d8d-4171-9ae9-a8b770cdc38b None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:59.456155 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:59.518414 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80f4151-4d8d-4171-9ae9-a8b770cdc38b None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:59.554715 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6856af40-2965-4e61-87cc-213428728bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:59.555752 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6856af40-2965-4e61-87cc-213428728bd3 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:59.556202 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6856af40-2965-4e61-87cc-213428728bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:59.557256 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6856af40-2965-4e61-87cc-213428728bd3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:59.567516 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6856af40-2965-4e61-87cc-213428728bd3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:59.569242 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6856af40-2965-4e61-87cc-213428728bd3 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:59.584124 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-6856af40-2965-4e61-87cc-213428728bd3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:20:59.584124 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:59.584124 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:59.584124 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:59.584124 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:59.584124 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:59.584124 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:59.584124 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:59.584124 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:59.584124 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:59.584124 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:59.584124 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:59.584124 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 31 12:20:59.584124 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 31 12:20:59.584124 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 31 12:20:59.584124 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 31 12:20:59.585100 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:59.585100 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:59.585100 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:59.585100 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:59.585100 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:59.585100 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:59.585100 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:59.585100 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:59.585100 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:59.585100 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:59.585100 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:59.585100 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:59.585100 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:59.585100 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:59.585100 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:59.585100 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:59.585100 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 31 12:20:59.585973 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 31 12:20:59.585973 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:20:59.585973 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:20:59.587736 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 271/555] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Oct 31 12:20:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:20:59.593407 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d28f44-ddaf-40e3-b446-305a05a50681 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:59.606055 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6856af40-2965-4e61-87cc-213428728bd3 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:59.662711 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:59.783440 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d28f44-ddaf-40e3-b446-305a05a50681 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:59.784805 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6856af40-2965-4e61-87cc-213428728bd3 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:20:59.786535 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:20:59.842297 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd0bccd-a178-4edf-9335-126a900524fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:59.842933 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd0bccd-a178-4edf-9335-126a900524fb None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:59.843225 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd0bccd-a178-4edf-9335-126a900524fb None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:59.843495 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd0bccd-a178-4edf-9335-126a900524fb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:59.850004 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92dc53b-18c0-48ec-ae79-ccc7ec52978c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:20:59.850615 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92dc53b-18c0-48ec-ae79-ccc7ec52978c None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:20:59.850888 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92dc53b-18c0-48ec-ae79-ccc7ec52978c None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:20:59.851156 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92dc53b-18c0-48ec-ae79-ccc7ec52978c None admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:20:59.851819 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92dc53b-18c0-48ec-ae79-ccc7ec52978c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:20:59.852115 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92dc53b-18c0-48ec-ae79-ccc7ec52978c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:20:59.853662 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92dc53b-18c0-48ec-ae79-ccc7ec52978c None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:20:59.869879 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd0bccd-a178-4edf-9335-126a900524fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 31 12:20:59.876899 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-3cd0bccd-a178-4edf-9335-126a900524fb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 31 12:20:59.876899 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:20:59.876899 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:20:59.876899 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:20:59.876899 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:20:59.876899 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:20:59.876899 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:20:59.876899 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:20:59.876899 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:20:59.876899 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:20:59.876899 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:20:59.876899 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:20:59.876899 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 31 12:20:59.876899 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 31 12:20:59.876899 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 31 12:20:59.876899 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 31 12:20:59.877807 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 31 12:20:59.877807 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 31 12:20:59.877807 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 31 12:20:59.877807 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 31 12:20:59.877807 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:20:59.877807 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:20:59.877807 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:20:59.877807 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:20:59.877807 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:20:59.877807 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:20:59.877807 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:20:59.877807 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:20:59.877807 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:20:59.877807 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:20:59.877807 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:20:59.877807 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:20:59.878678 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:20:59.878678 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:20:59.878678 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:20:59.878678 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:20:59.878678 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 31 12:20:59.878678 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 31 12:20:59.878678 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 31 12:20:59.878678 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 31 12:20:59.878678 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 31 12:20:59.878678 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 31 12:20:59.878678 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 31 12:20:59.878678 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:20:59.883364 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 272/556] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Oct 31 12:20:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 31 12:20:59.883601 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 285/557] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Oct 31 12:20:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:20:59.885757 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92dc53b-18c0-48ec-ae79-ccc7ec52978c None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:20:59.889651 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd0bccd-a178-4edf-9335-126a900524fb None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:00.019478 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:00.023495 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92dc53b-18c0-48ec-ae79-ccc7ec52978c None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:00.069369 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc6fdd8-5293-4401-b259-ae1c727cdd6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:00.070755 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc6fdd8-5293-4401-b259-ae1c727cdd6c None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:00.071277 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc6fdd8-5293-4401-b259-ae1c727cdd6c None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:00.071767 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc6fdd8-5293-4401-b259-ae1c727cdd6c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:00.079851 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc6fdd8-5293-4401-b259-ae1c727cdd6c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:00.081611 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc6fdd8-5293-4401-b259-ae1c727cdd6c None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:00.094498 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-7dc6fdd8-5293-4401-b259-ae1c727cdd6c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 31 12:21:00.094498 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:00.094498 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:21:00.094498 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:21:00.094498 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:21:00.094498 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:21:00.094498 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:21:00.094498 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:21:00.094498 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:21:00.094498 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:21:00.094498 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:21:00.094498 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:21:00.094498 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 31 12:21:00.094498 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 31 12:21:00.094498 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 31 12:21:00.094498 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 31 12:21:00.095539 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:21:00.095539 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:21:00.095539 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 31 12:21:00.095539 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 31 12:21:00.095539 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 31 12:21:00.095539 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 31 12:21:00.095539 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:21:00.095539 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:21:00.095539 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:21:00.095539 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:21:00.095539 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:21:00.095539 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 31 12:21:00.095539 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:21:00.095539 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:21:00.095539 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:21:00.095539 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:21:00.095539 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:21:00.095539 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:21:00.096466 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:21:00.096466 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:21:00.096466 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 31 12:21:00.096466 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 31 12:21:00.096466 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 31 12:21:00.096466 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 31 12:21:00.096466 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 31 12:21:00.096466 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 31 12:21:00.096466 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 31 12:21:00.096466 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:21:00.098680 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 286/558] 10.222.0.15 () {56 vars in 1117 bytes} [Mon Oct 31 12:20:59 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:21:00.102915 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd0bccd-a178-4edf-9335-126a900524fb None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:00.105216 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc6fdd8-5293-4401-b259-ae1c727cdd6c None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:00.188639 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e05bd6-f33d-4a8e-a15e-07f9671e6cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:00.189329 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e05bd6-f33d-4a8e-a15e-07f9671e6cef None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:00.189615 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e05bd6-f33d-4a8e-a15e-07f9671e6cef None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:00.189889 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e05bd6-f33d-4a8e-a15e-07f9671e6cef None admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:00.190686 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e05bd6-f33d-4a8e-a15e-07f9671e6cef None admin] RBAC: Adding query filter params (name=admin) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:00.190859 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e05bd6-f33d-4a8e-a15e-07f9671e6cef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:00.192263 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e05bd6-f33d-4a8e-a15e-07f9671e6cef None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:00.209921 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 273/559] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Oct 31 12:20:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:21:00.215329 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e05bd6-f33d-4a8e-a15e-07f9671e6cef None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:00.218046 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc6fdd8-5293-4401-b259-ae1c727cdd6c None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:00.250157 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:00.255923 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c66290-8fca-444a-8170-99576da58e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:00.256828 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c66290-8fca-444a-8170-99576da58e89 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:00.257354 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c66290-8fca-444a-8170-99576da58e89 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:00.257753 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c66290-8fca-444a-8170-99576da58e89 None admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:00.258607 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c66290-8fca-444a-8170-99576da58e89 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:00.259057 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c66290-8fca-444a-8170-99576da58e89 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:00.261436 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c66290-8fca-444a-8170-99576da58e89 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:00.282588 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 287/560] 10.222.0.15 () {56 vars in 1149 bytes} [Mon Oct 31 12:21:00 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:21:00.288692 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214e6cf8-c2cb-4899-a78f-a4fcd29f00f6 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:00.289278 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214e6cf8-c2cb-4899-a78f-a4fcd29f00f6 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:00.289724 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214e6cf8-c2cb-4899-a78f-a4fcd29f00f6 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:00.291774 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 288/561] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:21:00 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:21:00.292631 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214e6cf8-c2cb-4899-a78f-a4fcd29f00f6 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:00.312619 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:00.348320 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e05bd6-f33d-4a8e-a15e-07f9671e6cef None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:00.387689 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8865a4-b927-4513-8967-b76ea9d819f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:00.388847 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8865a4-b927-4513-8967-b76ea9d819f2 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:00.389677 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8865a4-b927-4513-8967-b76ea9d819f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:00.390127 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8865a4-b927-4513-8967-b76ea9d819f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:00.404730 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8865a4-b927-4513-8967-b76ea9d819f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:00.405575 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214e6cf8-c2cb-4899-a78f-a4fcd29f00f6 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:00.406451 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8865a4-b927-4513-8967-b76ea9d819f2 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:00.425085 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-bc8865a4-b927-4513-8967-b76ea9d819f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:21:00.425085 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:00.425085 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:21:00.425085 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:21:00.425085 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:21:00.425085 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:21:00.425085 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:21:00.425085 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:21:00.425085 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:21:00.425085 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:21:00.425085 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:21:00.425085 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:21:00.425085 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 31 12:21:00.425085 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 31 12:21:00.425085 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 31 12:21:00.425085 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 31 12:21:00.426230 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:21:00.426230 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:21:00.426230 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:21:00.426230 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:21:00.426230 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:21:00.426230 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:21:00.426230 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:21:00.426230 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:21:00.426230 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:21:00.426230 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:21:00.426230 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:21:00.426230 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:21:00.426230 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:21:00.426230 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:21:00.426230 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:21:00.426230 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:21:00.426230 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 31 12:21:00.427365 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 31 12:21:00.427365 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:21:00.427365 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:21:00.427878 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 274/562] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Oct 31 12:21:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:21:00.429295 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47461d51-1e36-4df7-9841-4ed470a63b4c None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:00.429574 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47461d51-1e36-4df7-9841-4ed470a63b4c None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:00.429842 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47461d51-1e36-4df7-9841-4ed470a63b4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:00.448775 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922e6c46-b12b-4a41-bf21-b55f45abbf81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:00.449629 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922e6c46-b12b-4a41-bf21-b55f45abbf81 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:00.449996 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922e6c46-b12b-4a41-bf21-b55f45abbf81 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:00.450417 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922e6c46-b12b-4a41-bf21-b55f45abbf81 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:00.453828 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-47461d51-1e36-4df7-9841-4ed470a63b4c None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:21:00.455180 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-47461d51-1e36-4df7-9841-4ed470a63b4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:21:00.467555 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922e6c46-b12b-4a41-bf21-b55f45abbf81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:00.469627 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922e6c46-b12b-4a41-bf21-b55f45abbf81 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:00.489279 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-922e6c46-b12b-4a41-bf21-b55f45abbf81 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:21:00.489279 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:00.489279 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:21:00.489279 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:21:00.489279 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:21:00.489279 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:21:00.489279 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:21:00.489279 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:21:00.489279 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:21:00.489279 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:21:00.489279 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:21:00.489279 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:21:00.489279 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 31 12:21:00.489279 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 31 12:21:00.489279 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 31 12:21:00.489279 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 31 12:21:00.490323 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:21:00.490323 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:21:00.490323 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:21:00.490323 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:21:00.490323 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:21:00.490323 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:21:00.490323 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:21:00.490323 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 31 12:21:00.490323 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:21:00.490323 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:21:00.490323 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:21:00.490323 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:21:00.490323 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:21:00.490323 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:21:00.490323 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:21:00.490323 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:21:00.490323 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 31 12:21:00.491230 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 31 12:21:00.491230 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:21:00.491230 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:21:00.495239 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 289/563] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Oct 31 12:21:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:21:00.497306 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922e6c46-b12b-4a41-bf21-b55f45abbf81 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:00.553948 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 275/564] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:21:00 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 31 12:21:00.555264 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47461d51-1e36-4df7-9841-4ed470a63b4c None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:00.559622 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:00.596516 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:00.633316 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922e6c46-b12b-4a41-bf21-b55f45abbf81 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:00.672804 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa5a06f-9fb9-4855-a9fd-d80b05476ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:00.673803 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa5a06f-9fb9-4855-a9fd-d80b05476ea7 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:00.674478 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa5a06f-9fb9-4855-a9fd-d80b05476ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:00.674952 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa5a06f-9fb9-4855-a9fd-d80b05476ea7 None admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:00.675806 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa5a06f-9fb9-4855-a9fd-d80b05476ea7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:00.676305 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa5a06f-9fb9-4855-a9fd-d80b05476ea7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:00.677861 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa5a06f-9fb9-4855-a9fd-d80b05476ea7 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:00.692435 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 290/565] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Oct 31 12:21:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:21:00.697754 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa5a06f-9fb9-4855-a9fd-d80b05476ea7 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:00.757452 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47461d51-1e36-4df7-9841-4ed470a63b4c None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:00.855889 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e97d7f5-8bab-4125-a7bc-5a1028db0e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:00.856563 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e97d7f5-8bab-4125-a7bc-5a1028db0e91 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:00.856802 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e97d7f5-8bab-4125-a7bc-5a1028db0e91 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:00.857047 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e97d7f5-8bab-4125-a7bc-5a1028db0e91 None admin] PATH_INFO: `/v3/domains` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:00.857673 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e97d7f5-8bab-4125-a7bc-5a1028db0e91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:00.857972 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e97d7f5-8bab-4125-a7bc-5a1028db0e91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:00.859023 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa5a06f-9fb9-4855-a9fd-d80b05476ea7 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:00.859356 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e97d7f5-8bab-4125-a7bc-5a1028db0e91 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:00.880639 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 276/566] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Oct 31 12:21:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:21:00.886609 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e97d7f5-8bab-4125-a7bc-5a1028db0e91 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:00.892585 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:00.900197 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a70e5b5-5e4e-4fdb-8225-f5b76fdafc61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:00.901336 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a70e5b5-5e4e-4fdb-8225-f5b76fdafc61 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:00.901811 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a70e5b5-5e4e-4fdb-8225-f5b76fdafc61 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:00.902305 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a70e5b5-5e4e-4fdb-8225-f5b76fdafc61 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:00.910740 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a70e5b5-5e4e-4fdb-8225-f5b76fdafc61 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:00.912513 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a70e5b5-5e4e-4fdb-8225-f5b76fdafc61 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:00.925850 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-7a70e5b5-5e4e-4fdb-8225-f5b76fdafc61 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 31 12:21:00.925850 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:00.925850 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:21:00.925850 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:21:00.925850 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:21:00.925850 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:21:00.925850 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:21:00.925850 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:21:00.925850 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:21:00.925850 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:21:00.925850 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:21:00.925850 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:21:00.925850 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 31 12:21:00.925850 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 31 12:21:00.925850 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 31 12:21:00.925850 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 31 12:21:00.926835 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:21:00.926835 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:21:00.926835 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 31 12:21:00.926835 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 31 12:21:00.926835 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 31 12:21:00.926835 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 31 12:21:00.926835 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:21:00.926835 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:21:00.926835 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:21:00.926835 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:21:00.926835 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:21:00.926835 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 31 12:21:00.926835 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:21:00.926835 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:21:00.926835 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:21:00.926835 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:21:00.926835 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:21:00.926835 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:21:00.927798 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:21:00.927798 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:21:00.927798 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 31 12:21:00.927798 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 31 12:21:00.927798 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 31 12:21:00.927798 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 31 12:21:00.927798 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 31 12:21:00.927798 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 31 12:21:00.927798 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 31 12:21:00.927798 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:21:00.930321 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 291/567] 10.222.0.15 () {56 vars in 1093 bytes} [Mon Oct 31 12:21:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:21:00.931482 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a70e5b5-5e4e-4fdb-8225-f5b76fdafc61 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:00.952406 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:00.999144 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e97d7f5-8bab-4125-a7bc-5a1028db0e91 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:01.041656 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b65bd0e-0086-4d4e-95fa-91fa9b9d7145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:01.042885 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b65bd0e-0086-4d4e-95fa-91fa9b9d7145 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:01.043310 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b65bd0e-0086-4d4e-95fa-91fa9b9d7145 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:01.043909 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b65bd0e-0086-4d4e-95fa-91fa9b9d7145 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:01.052568 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a70e5b5-5e4e-4fdb-8225-f5b76fdafc61 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:01.054442 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b65bd0e-0086-4d4e-95fa-91fa9b9d7145 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:01.057387 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b65bd0e-0086-4d4e-95fa-91fa9b9d7145 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:01.075246 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-2b65bd0e-0086-4d4e-95fa-91fa9b9d7145 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 31 12:21:01.075246 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:01.075246 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:21:01.075246 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:21:01.075246 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:21:01.075246 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:21:01.075246 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:21:01.075246 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:21:01.075246 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:21:01.075246 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:21:01.075246 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:21:01.075246 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:21:01.075246 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 31 12:21:01.075246 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 31 12:21:01.075246 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 31 12:21:01.076217 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 31 12:21:01.076217 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:21:01.076217 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:21:01.076217 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:21:01.076217 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:21:01.076217 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:21:01.076217 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:21:01.076217 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:21:01.076217 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:21:01.076217 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:21:01.076217 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:21:01.076217 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:21:01.076217 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:21:01.076217 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:21:01.076217 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:21:01.076217 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:21:01.076217 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:21:01.077194 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 31 12:21:01.077194 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 31 12:21:01.077194 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 31 12:21:01.077194 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 31 12:21:01.077194 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 31 12:21:01.077194 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 31 12:21:01.077194 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 31 12:21:01.077194 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:21:01.079292 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 277/568] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Oct 31 12:21:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 31 12:21:01.080340 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b65bd0e-0086-4d4e-95fa-91fa9b9d7145 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:01.100005 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc80ee9-6c53-4aa8-b2cf-3f8f8d2d2927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:01.100798 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc80ee9-6c53-4aa8-b2cf-3f8f8d2d2927 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:01.101182 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc80ee9-6c53-4aa8-b2cf-3f8f8d2d2927 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:01.101582 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc80ee9-6c53-4aa8-b2cf-3f8f8d2d2927 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:01.102274 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc80ee9-6c53-4aa8-b2cf-3f8f8d2d2927 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:01.105080 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc80ee9-6c53-4aa8-b2cf-3f8f8d2d2927 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:01.115766 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-6cc80ee9-6c53-4aa8-b2cf-3f8f8d2d2927 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 31 12:21:01.115766 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:01.115766 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 31 12:21:01.115766 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 31 12:21:01.115766 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 31 12:21:01.115766 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 31 12:21:01.115766 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 31 12:21:01.115766 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 31 12:21:01.115766 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 31 12:21:01.115766 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:21:01.115766 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 31 12:21:01.115766 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:21:01.115766 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:01.115766 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:21:01.115766 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:21:01.115766 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:21:01.115766 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:21:01.117681 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:21:01.117681 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:21:01.117681 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:21:01.117681 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:21:01.117681 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:21:01.117681 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:21:01.117681 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 31 12:21:01.117681 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 31 12:21:01.117681 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 31 12:21:01.117681 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 31 12:21:01.117681 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:21:01.117681 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:21:01.117681 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:21:01.117681 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:21:01.117681 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:21:01.117681 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:21:01.119383 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:21:01.119383 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 31 12:21:01.119383 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:21:01.119383 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:21:01.119383 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:21:01.119383 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:21:01.119383 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:21:01.119383 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:21:01.119383 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:21:01.119383 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:21:01.119383 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 31 12:21:01.119383 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 31 12:21:01.119383 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 31 12:21:01.119383 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:21:01.127218 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 292/569] 10.222.0.15 () {56 vars in 1129 bytes} [Mon Oct 31 12:21:00 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:21:01.127218 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc80ee9-6c53-4aa8-b2cf-3f8f8d2d2927 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:01.196381 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:01.214263 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b65bd0e-0086-4d4e-95fa-91fa9b9d7145 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:01.237868 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:01.250030 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc80ee9-6c53-4aa8-b2cf-3f8f8d2d2927 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:01.250860 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ebbd55-4d39-462c-81a4-81510f662552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:01.251504 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ebbd55-4d39-462c-81a4-81510f662552 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:01.251787 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ebbd55-4d39-462c-81a4-81510f662552 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:01.252059 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ebbd55-4d39-462c-81a4-81510f662552 None admin] PATH_INFO: `/v3/users` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:01.253072 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ebbd55-4d39-462c-81a4-81510f662552 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:01.253072 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ebbd55-4d39-462c-81a4-81510f662552 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:01.254486 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ebbd55-4d39-462c-81a4-81510f662552 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:01.281173 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 278/570] 10.222.0.15 () {56 vars in 1133 bytes} [Mon Oct 31 12:21:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:21:01.283132 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ebbd55-4d39-462c-81a4-81510f662552 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:01.290583 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb25101-761a-4d40-8705-9d05ea72f558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:01.291317 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb25101-761a-4d40-8705-9d05ea72f558 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:01.291573 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb25101-761a-4d40-8705-9d05ea72f558 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:01.291881 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb25101-761a-4d40-8705-9d05ea72f558 None admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:01.292606 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb25101-761a-4d40-8705-9d05ea72f558 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:01.292909 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb25101-761a-4d40-8705-9d05ea72f558 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:01.294317 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb25101-761a-4d40-8705-9d05ea72f558 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:01.307476 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 293/571] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Oct 31 12:21:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:21:01.308910 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca79ea0-6f78-42c5-a419-a2889f409c48 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:01.309198 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca79ea0-6f78-42c5-a419-a2889f409c48 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:01.309468 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca79ea0-6f78-42c5-a419-a2889f409c48 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:01.314624 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 294/572] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:21:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:21:01.314624 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca79ea0-6f78-42c5-a419-a2889f409c48 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:01.399376 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ebbd55-4d39-462c-81a4-81510f662552 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:01.421389 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca79ea0-6f78-42c5-a419-a2889f409c48 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:01.436672 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0f9998-5a44-4e31-94ce-3b52a8565b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:01.437283 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0f9998-5a44-4e31-94ce-3b52a8565b07 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:01.437557 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0f9998-5a44-4e31-94ce-3b52a8565b07 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:01.437823 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0f9998-5a44-4e31-94ce-3b52a8565b07 None admin] PATH_INFO: `/v3/services` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:01.437928 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:01.438448 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0f9998-5a44-4e31-94ce-3b52a8565b07 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:01.438751 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0f9998-5a44-4e31-94ce-3b52a8565b07 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:01.440185 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0f9998-5a44-4e31-94ce-3b52a8565b07 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:01.444940 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 279/573] 10.222.0.15 () {56 vars in 1124 bytes} [Mon Oct 31 12:21:01 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:21:01.444940 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0f9998-5a44-4e31-94ce-3b52a8565b07 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:01.469021 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52402112-cf54-49cc-911b-d115c07970fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:01.469565 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52402112-cf54-49cc-911b-d115c07970fb None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:01.469842 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52402112-cf54-49cc-911b-d115c07970fb None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:01.470112 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52402112-cf54-49cc-911b-d115c07970fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:01.499283 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:01.546024 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52402112-cf54-49cc-911b-d115c07970fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:01.547490 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52402112-cf54-49cc-911b-d115c07970fb None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:01.556041 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0f9998-5a44-4e31-94ce-3b52a8565b07 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:01.591876 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9947fed-f483-4128-a3b4-ec52d46666fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:01.592533 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9947fed-f483-4128-a3b4-ec52d46666fc None admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:01.592808 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9947fed-f483-4128-a3b4-ec52d46666fc None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:01.593077 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9947fed-f483-4128-a3b4-ec52d46666fc None admin] PATH_INFO: `/v3/projects/2c32444cef584bce966b38203ce128b2/users/bf4faacf7df248d28f32d7014dadcc07/roles/9674a2c5acc74dfeb93cfd67dfd890dd` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:01.607771 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9947fed-f483-4128-a3b4-ec52d46666fc None admin] RBAC: Authorizing `identity:create_grant(project_id=2c32444cef584bce966b38203ce128b2, user_id=bf4faacf7df248d28f32d7014dadcc07, role_id=9674a2c5acc74dfeb93cfd67dfd890dd)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:01.609181 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9947fed-f483-4128-a3b4-ec52d46666fc None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:01.621879 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 280/574] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Oct 31 12:21:01 2022] PUT /identity/v3/projects/2c32444cef584bce966b38203ce128b2/users/bf4faacf7df248d28f32d7014dadcc07/roles/9674a2c5acc74dfeb93cfd67dfd890dd => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:21:01.622877 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bda05a5-5969-4472-9019-b478188d69b2 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:01.623187 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bda05a5-5969-4472-9019-b478188d69b2 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:01.623457 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bda05a5-5969-4472-9019-b478188d69b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:01.642546 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-7bda05a5-5969-4472-9019-b478188d69b2 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:21:01.644056 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-7bda05a5-5969-4472-9019-b478188d69b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:21:01.663221 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 295/575] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Oct 31 12:21:01 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 31 12:21:01.663993 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52402112-cf54-49cc-911b-d115c07970fb None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:01.682650 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:01.755276 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 281/576] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:21:01 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 31 12:21:01.755276 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761cc498-03f4-4f44-a37b-2203dfccb89c None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:01.755276 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761cc498-03f4-4f44-a37b-2203dfccb89c None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:01.755276 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761cc498-03f4-4f44-a37b-2203dfccb89c None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:01.755276 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 282/577] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:21:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:21:01.755276 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761cc498-03f4-4f44-a37b-2203dfccb89c None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:01.787121 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:01.800108 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52402112-cf54-49cc-911b-d115c07970fb None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:01.858961 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933696e3-8bf6-4279-bc46-229d7ed90a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:01.859693 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933696e3-8bf6-4279-bc46-229d7ed90a1a None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:01.859974 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933696e3-8bf6-4279-bc46-229d7ed90a1a None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:01.860246 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933696e3-8bf6-4279-bc46-229d7ed90a1a None admin] PATH_INFO: `/v3/services` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:01.861001 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933696e3-8bf6-4279-bc46-229d7ed90a1a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:01.861478 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933696e3-8bf6-4279-bc46-229d7ed90a1a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:01.862908 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933696e3-8bf6-4279-bc46-229d7ed90a1a None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:01.871251 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 296/578] 10.222.0.15 () {56 vars in 1124 bytes} [Mon Oct 31 12:21:01 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:21:01.872876 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5486e5f-1600-4bff-9693-627737d75c48 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:01.873363 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5486e5f-1600-4bff-9693-627737d75c48 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:01.873819 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5486e5f-1600-4bff-9693-627737d75c48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:01.902057 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-d5486e5f-1600-4bff-9693-627737d75c48 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:21:01.903290 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-d5486e5f-1600-4bff-9693-627737d75c48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:21:01.907240 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761cc498-03f4-4f44-a37b-2203dfccb89c None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:01.957933 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be62518-0452-460c-9113-dcfe1d4d51b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:01.959120 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be62518-0452-460c-9113-dcfe1d4d51b0 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:01.959701 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be62518-0452-460c-9113-dcfe1d4d51b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:01.960278 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be62518-0452-460c-9113-dcfe1d4d51b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:01.972820 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be62518-0452-460c-9113-dcfe1d4d51b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:01.974534 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be62518-0452-460c-9113-dcfe1d4d51b0 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:01.988580 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 297/579] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:21:01 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 31 12:21:01.990207 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5486e5f-1600-4bff-9693-627737d75c48 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:01.997702 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-1be62518-0452-460c-9113-dcfe1d4d51b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:21:01.997702 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:01.997702 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:21:01.997702 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:21:01.997702 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:21:01.997702 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:21:01.997702 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:21:01.997702 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:21:01.997702 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:21:01.997702 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:21:01.997702 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:21:01.997702 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:21:01.997702 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 31 12:21:01.997702 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 31 12:21:01.997702 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 31 12:21:01.997702 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 31 12:21:01.999147 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:21:01.999147 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:21:01.999147 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:21:01.999147 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:21:01.999147 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:21:01.999147 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:21:01.999147 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:21:01.999147 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:21:01.999147 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:21:01.999147 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:21:01.999147 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:21:01.999147 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:21:01.999147 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:21:01.999147 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:21:01.999147 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:21:01.999147 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:21:01.999147 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 31 12:21:02.001002 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 31 12:21:02.001002 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:21:02.001002 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:21:02.003239 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:02.005380 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 283/580] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Oct 31 12:21:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:21:02.011437 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be62518-0452-460c-9113-dcfe1d4d51b0 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:02.167571 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:02.181816 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5486e5f-1600-4bff-9693-627737d75c48 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:02.195050 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be62518-0452-460c-9113-dcfe1d4d51b0 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:02.220449 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf69b22-2b88-4d9a-a59c-f265097ce241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:02.221391 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf69b22-2b88-4d9a-a59c-f265097ce241 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:02.221675 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf69b22-2b88-4d9a-a59c-f265097ce241 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:02.221946 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf69b22-2b88-4d9a-a59c-f265097ce241 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:02.222921 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf69b22-2b88-4d9a-a59c-f265097ce241 None admin] RBAC: Adding query filter params (interface=public, service_id=9cd1592d26fa4d55b36f7113feed5236, region_id=RegionOne) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:02.223259 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf69b22-2b88-4d9a-a59c-f265097ce241 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:02.224643 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf69b22-2b88-4d9a-a59c-f265097ce241 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:02.233867 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9339d3-e017-4d25-89f7-65b46a2a1ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:02.234758 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9339d3-e017-4d25-89f7-65b46a2a1ec2 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:02.235275 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9339d3-e017-4d25-89f7-65b46a2a1ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:02.235762 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9339d3-e017-4d25-89f7-65b46a2a1ec2 None admin] PATH_INFO: `/v3/domains` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:02.236611 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9339d3-e017-4d25-89f7-65b46a2a1ec2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:02.237268 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9339d3-e017-4d25-89f7-65b46a2a1ec2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:02.238846 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9339d3-e017-4d25-89f7-65b46a2a1ec2 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:02.239404 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 298/581] 10.222.0.15 () {56 vars in 1251 bytes} [Mon Oct 31 12:21:01 2022] GET /identity/v3/endpoints?service_id=9cd1592d26fa4d55b36f7113feed5236&interface=public®ion_id=RegionOne => generated 194 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:21:02.240281 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf69b22-2b88-4d9a-a59c-f265097ce241 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:02.253627 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 284/582] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Oct 31 12:21:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 31 12:21:02.254841 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9339d3-e017-4d25-89f7-65b46a2a1ec2 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:02.329404 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:02.367777 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf69b22-2b88-4d9a-a59c-f265097ce241 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:02.371180 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9339d3-e017-4d25-89f7-65b46a2a1ec2 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:02.404680 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb0c365-8480-424d-8ce4-b7a1ffa69212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:02.405317 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb0c365-8480-424d-8ce4-b7a1ffa69212 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:02.405602 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb0c365-8480-424d-8ce4-b7a1ffa69212 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:02.405874 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb0c365-8480-424d-8ce4-b7a1ffa69212 None admin] PATH_INFO: `/v3/services/network` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:02.406449 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb0c365-8480-424d-8ce4-b7a1ffa69212 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:02.407841 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb0c365-8480-424d-8ce4-b7a1ffa69212 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:02.408568 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1d3931-1316-4245-be78-7965b260dfb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:02.409326 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1d3931-1316-4245-be78-7965b260dfb5 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:02.409757 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1d3931-1316-4245-be78-7965b260dfb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:02.410179 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1d3931-1316-4245-be78-7965b260dfb5 None admin] PATH_INFO: `/v3/services` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:02.410926 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1d3931-1316-4245-be78-7965b260dfb5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:02.412449 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1d3931-1316-4245-be78-7965b260dfb5 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:02.414709 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:02.417776 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-4bb0c365-8480-424d-8ce4-b7a1ffa69212 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 31 12:21:02.417776 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:02.417776 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 31 12:21:02.417776 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 31 12:21:02.417776 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 31 12:21:02.417776 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 31 12:21:02.417776 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 31 12:21:02.417776 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 31 12:21:02.417776 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 31 12:21:02.417776 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:21:02.417776 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 31 12:21:02.417776 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:21:02.417776 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:02.417776 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:21:02.417776 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:21:02.417776 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:21:02.417776 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:21:02.418965 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:21:02.418965 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:21:02.418965 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:21:02.418965 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:21:02.418965 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:21:02.418965 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:21:02.418965 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 31 12:21:02.418965 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 31 12:21:02.418965 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 31 12:21:02.418965 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 31 12:21:02.418965 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:21:02.418965 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:21:02.418965 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:21:02.418965 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:21:02.418965 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:21:02.418965 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:21:02.420166 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:21:02.420166 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 31 12:21:02.420166 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:21:02.420166 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:21:02.420166 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:21:02.420166 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:21:02.420166 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:21:02.420166 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:21:02.420166 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:21:02.420166 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:21:02.420166 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 31 12:21:02.420166 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 31 12:21:02.420166 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 31 12:21:02.420166 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:21:02.423597 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 285/583] 10.222.0.15 () {56 vars in 1087 bytes} [Mon Oct 31 12:21:02 2022] GET /identity/v3/services => generated 772 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:21:02.423725 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 299/584] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Oct 31 12:21:02 2022] GET /identity/v3/services/network => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:21:02.424697 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1d3931-1316-4245-be78-7965b260dfb5 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:02.425417 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb0c365-8480-424d-8ce4-b7a1ffa69212 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:02.548081 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1d3931-1316-4245-be78-7965b260dfb5 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:02.595791 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4550b11-733f-461c-a4b8-5c8de1704847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:02.596695 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4550b11-733f-461c-a4b8-5c8de1704847 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:02.597218 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4550b11-733f-461c-a4b8-5c8de1704847 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:02.597688 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4550b11-733f-461c-a4b8-5c8de1704847 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:02.606621 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb0c365-8480-424d-8ce4-b7a1ffa69212 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:02.612746 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4550b11-733f-461c-a4b8-5c8de1704847 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 31 12:21:02.617657 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-d4550b11-733f-461c-a4b8-5c8de1704847 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 31 12:21:02.617657 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:02.617657 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:21:02.617657 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:21:02.617657 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:21:02.617657 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:21:02.617657 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:21:02.617657 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:21:02.617657 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:21:02.617657 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:21:02.617657 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:21:02.617657 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:21:02.617657 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 31 12:21:02.617657 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 31 12:21:02.617657 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 31 12:21:02.617657 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 31 12:21:02.618632 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 31 12:21:02.618632 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 31 12:21:02.618632 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 31 12:21:02.618632 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 31 12:21:02.618632 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:21:02.618632 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:21:02.618632 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:21:02.618632 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:21:02.618632 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:21:02.618632 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:21:02.618632 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:21:02.618632 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:21:02.618632 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:21:02.618632 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:21:02.618632 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:21:02.618632 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:21:02.621335 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:21:02.621335 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:21:02.621335 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:21:02.621335 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:21:02.621335 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 31 12:21:02.621335 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 31 12:21:02.621335 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 31 12:21:02.621335 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 31 12:21:02.621335 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 31 12:21:02.621335 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 31 12:21:02.621335 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 31 12:21:02.621335 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:21:02.621335 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 286/585] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Oct 31 12:21:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:21:02.622049 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4550b11-733f-461c-a4b8-5c8de1704847 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:02.654655 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8a212d-e99e-480e-a7fc-ac7cbf5e56ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:02.655944 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8a212d-e99e-480e-a7fc-ac7cbf5e56ff None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:02.656346 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8a212d-e99e-480e-a7fc-ac7cbf5e56ff None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:02.656707 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8a212d-e99e-480e-a7fc-ac7cbf5e56ff None admin] PATH_INFO: `/v3/projects/service` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:02.659774 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:02.665575 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8a212d-e99e-480e-a7fc-ac7cbf5e56ff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:02.668042 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8a212d-e99e-480e-a7fc-ac7cbf5e56ff None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:02.680963 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-cd8a212d-e99e-480e-a7fc-ac7cbf5e56ff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 31 12:21:02.680963 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:02.680963 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:21:02.680963 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:21:02.680963 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:21:02.680963 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:21:02.680963 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:21:02.680963 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:21:02.680963 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:21:02.680963 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:21:02.680963 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:21:02.680963 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:21:02.680963 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 31 12:21:02.680963 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 31 12:21:02.680963 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 31 12:21:02.682618 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 31 12:21:02.682618 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:21:02.682618 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:21:02.682618 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:21:02.682618 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:21:02.682618 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:21:02.682618 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:21:02.682618 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:21:02.682618 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 31 12:21:02.682618 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:21:02.682618 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:21:02.682618 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:21:02.682618 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:21:02.682618 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:21:02.682618 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:21:02.682618 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:21:02.682618 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:21:02.683758 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 31 12:21:02.683758 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 31 12:21:02.683758 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 31 12:21:02.683758 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 31 12:21:02.683758 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 31 12:21:02.683758 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 31 12:21:02.683758 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 31 12:21:02.683758 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:21:02.685218 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 300/586] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Oct 31 12:21:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:21:02.686281 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8a212d-e99e-480e-a7fc-ac7cbf5e56ff None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:02.713781 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:02.768284 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4550b11-733f-461c-a4b8-5c8de1704847 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:02.794642 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8a212d-e99e-480e-a7fc-ac7cbf5e56ff None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:02.813186 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ca129f-7350-4901-b2c2-1fa01763f2c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:02.814079 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ca129f-7350-4901-b2c2-1fa01763f2c5 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:02.814360 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ca129f-7350-4901-b2c2-1fa01763f2c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:02.814630 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ca129f-7350-4901-b2c2-1fa01763f2c5 None admin] PATH_INFO: `/v3/services` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:02.815280 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ca129f-7350-4901-b2c2-1fa01763f2c5 None admin] RBAC: Adding query filter params (name=network) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:02.815586 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ca129f-7350-4901-b2c2-1fa01763f2c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:02.817401 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ca129f-7350-4901-b2c2-1fa01763f2c5 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:02.823925 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 287/587] 10.222.0.15 () {56 vars in 1112 bytes} [Mon Oct 31 12:21:02 2022] GET /identity/v3/services?name=network => generated 124 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:21:02.824822 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ca129f-7350-4901-b2c2-1fa01763f2c5 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:02.836202 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4f24f2-1ef9-4014-af37-fb1c1bc15a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:02.837015 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4f24f2-1ef9-4014-af37-fb1c1bc15a3c None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:02.837384 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4f24f2-1ef9-4014-af37-fb1c1bc15a3c None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:02.838542 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4f24f2-1ef9-4014-af37-fb1c1bc15a3c None admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:02.839367 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4f24f2-1ef9-4014-af37-fb1c1bc15a3c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:02.839765 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4f24f2-1ef9-4014-af37-fb1c1bc15a3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:02.841314 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4f24f2-1ef9-4014-af37-fb1c1bc15a3c None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:02.852618 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 301/588] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Oct 31 12:21:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:21:02.853703 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4f24f2-1ef9-4014-af37-fb1c1bc15a3c None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:02.927601 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:02.944015 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ca129f-7350-4901-b2c2-1fa01763f2c5 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:02.976672 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:02.976990 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4f24f2-1ef9-4014-af37-fb1c1bc15a3c None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:02.990168 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeca93ec-e2c4-4d74-9d8a-8ad1445e6974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:02.990915 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeca93ec-e2c4-4d74-9d8a-8ad1445e6974 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:02.991296 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeca93ec-e2c4-4d74-9d8a-8ad1445e6974 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:02.991639 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeca93ec-e2c4-4d74-9d8a-8ad1445e6974 None admin] PATH_INFO: `/v3/projects` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:02.992415 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeca93ec-e2c4-4d74-9d8a-8ad1445e6974 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:02.992872 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeca93ec-e2c4-4d74-9d8a-8ad1445e6974 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:02.995028 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeca93ec-e2c4-4d74-9d8a-8ad1445e6974 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:03.011545 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 288/589] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Oct 31 12:21:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:21:03.012546 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeca93ec-e2c4-4d74-9d8a-8ad1445e6974 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:03.014775 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9e07a3-734e-4588-b322-49bc8f254a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:03.015373 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9e07a3-734e-4588-b322-49bc8f254a0c None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:03.015648 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9e07a3-734e-4588-b322-49bc8f254a0c None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:03.015913 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9e07a3-734e-4588-b322-49bc8f254a0c None admin] PATH_INFO: `/v3/services` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:03.016532 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9e07a3-734e-4588-b322-49bc8f254a0c None admin] RBAC: Adding query filter params (type=network) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:03.016831 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9e07a3-734e-4588-b322-49bc8f254a0c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:03.019490 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9e07a3-734e-4588-b322-49bc8f254a0c None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:03.029542 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 302/590] 10.222.0.15 () {56 vars in 1112 bytes} [Mon Oct 31 12:21:02 2022] GET /identity/v3/services?type=network => generated 350 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:21:03.035890 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9e07a3-734e-4588-b322-49bc8f254a0c None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:03.136342 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeca93ec-e2c4-4d74-9d8a-8ad1445e6974 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:03.155006 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9e07a3-734e-4588-b322-49bc8f254a0c None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:03.173574 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76567df5-f02a-40eb-b8eb-b910cec7c07a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:03.174138 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76567df5-f02a-40eb-b8eb-b910cec7c07a None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:03.174474 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76567df5-f02a-40eb-b8eb-b910cec7c07a None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:03.174751 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76567df5-f02a-40eb-b8eb-b910cec7c07a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:03.182762 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:03.187752 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76567df5-f02a-40eb-b8eb-b910cec7c07a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:03.189348 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76567df5-f02a-40eb-b8eb-b910cec7c07a None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:03.196736 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd081643-b6a3-4737-8840-1de03e85e9ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:03.198566 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd081643-b6a3-4737-8840-1de03e85e9ea None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:03.199021 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd081643-b6a3-4737-8840-1de03e85e9ea None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:03.199950 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd081643-b6a3-4737-8840-1de03e85e9ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:03.201147 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd081643-b6a3-4737-8840-1de03e85e9ea None admin] RBAC: Adding query filter params (role.id=176c07b4fe3b4537bdfeb4be21b379c5, scope.project.id=2c32444cef584bce966b38203ce128b2, user.id=99889b1dab0b471d9c77887cffefc652) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:03.201629 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd081643-b6a3-4737-8840-1de03e85e9ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:03.203221 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd081643-b6a3-4737-8840-1de03e85e9ea None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:03.206479 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-76567df5-f02a-40eb-b8eb-b910cec7c07a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:21:03.206479 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:03.206479 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:21:03.206479 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:21:03.206479 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:21:03.206479 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:21:03.206479 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:21:03.206479 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:21:03.206479 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:21:03.206479 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:21:03.206479 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:21:03.206479 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:21:03.206479 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 31 12:21:03.206479 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 31 12:21:03.206479 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 31 12:21:03.206479 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 31 12:21:03.207435 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:21:03.207435 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:21:03.207435 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:21:03.207435 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:21:03.207435 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:21:03.207435 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:21:03.207435 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:21:03.207435 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:21:03.207435 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:21:03.207435 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:21:03.207435 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:21:03.207435 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:21:03.207435 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:21:03.207435 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:21:03.207435 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:21:03.207435 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:21:03.207435 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 31 12:21:03.208317 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 31 12:21:03.208317 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:21:03.208317 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:21:03.208317 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 289/591] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Oct 31 12:21:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:21:03.209202 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76567df5-f02a-40eb-b8eb-b910cec7c07a None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:03.212480 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 303/592] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Oct 31 12:21:03 2022] GET /identity/v3/role_assignments?user.id=99889b1dab0b471d9c77887cffefc652&scope.project.id=2c32444cef584bce966b38203ce128b2&role.id=176c07b4fe3b4537bdfeb4be21b379c5 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:21:03.213896 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd081643-b6a3-4737-8840-1de03e85e9ea None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:03.276974 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:03.330085 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd081643-b6a3-4737-8840-1de03e85e9ea None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:03.371895 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116e37d3-8cc5-4126-bd10-4266dfa1a6ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:03.372911 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116e37d3-8cc5-4126-bd10-4266dfa1a6ca None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:03.373463 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116e37d3-8cc5-4126-bd10-4266dfa1a6ca None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:03.373951 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116e37d3-8cc5-4126-bd10-4266dfa1a6ca None admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:03.374848 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116e37d3-8cc5-4126-bd10-4266dfa1a6ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:03.375367 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116e37d3-8cc5-4126-bd10-4266dfa1a6ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:03.377166 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116e37d3-8cc5-4126-bd10-4266dfa1a6ca None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:03.392446 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 304/593] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Oct 31 12:21:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:21:03.400404 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116e37d3-8cc5-4126-bd10-4266dfa1a6ca None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:03.411053 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76567df5-f02a-40eb-b8eb-b910cec7c07a None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:03.490330 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666e010a-fd4b-48bf-a0dc-2a812064f78a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:03.490966 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666e010a-fd4b-48bf-a0dc-2a812064f78a None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:03.491269 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666e010a-fd4b-48bf-a0dc-2a812064f78a None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:03.491541 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666e010a-fd4b-48bf-a0dc-2a812064f78a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:03.492250 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666e010a-fd4b-48bf-a0dc-2a812064f78a None admin] RBAC: Adding query filter params (interface=public, service_id=d3408d8979944b8c9d16e3d17575cc7e, region_id=RegionOne) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:03.492557 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666e010a-fd4b-48bf-a0dc-2a812064f78a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:03.494809 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666e010a-fd4b-48bf-a0dc-2a812064f78a None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:03.508274 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:03.512230 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 290/594] 10.222.0.15 () {56 vars in 1251 bytes} [Mon Oct 31 12:21:03 2022] GET /identity/v3/endpoints?service_id=d3408d8979944b8c9d16e3d17575cc7e&interface=public®ion_id=RegionOne => generated 194 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:21:03.518946 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666e010a-fd4b-48bf-a0dc-2a812064f78a None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:03.577352 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116e37d3-8cc5-4126-bd10-4266dfa1a6ca None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:03.615864 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182b2d2e-947b-4975-bfe5-329f4d8f731e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:03.616827 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182b2d2e-947b-4975-bfe5-329f4d8f731e None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:03.617320 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182b2d2e-947b-4975-bfe5-329f4d8f731e None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:03.617781 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182b2d2e-947b-4975-bfe5-329f4d8f731e None admin] PATH_INFO: `/v3/users/swift` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:03.626928 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182b2d2e-947b-4975-bfe5-329f4d8f731e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:03.628689 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182b2d2e-947b-4975-bfe5-329f4d8f731e None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:03.637057 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:03.640459 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666e010a-fd4b-48bf-a0dc-2a812064f78a None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:03.644492 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-182b2d2e-947b-4975-bfe5-329f4d8f731e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 31 12:21:03.644492 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:03.644492 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:21:03.644492 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:21:03.644492 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:21:03.644492 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:21:03.644492 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:21:03.644492 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:21:03.644492 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:21:03.644492 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:21:03.644492 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:21:03.644492 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:21:03.644492 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 31 12:21:03.644492 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 31 12:21:03.644492 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 31 12:21:03.644492 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 31 12:21:03.645568 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:21:03.645568 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:21:03.645568 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 31 12:21:03.645568 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 31 12:21:03.645568 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 31 12:21:03.645568 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 31 12:21:03.645568 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:21:03.645568 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:21:03.645568 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:21:03.645568 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:21:03.645568 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:21:03.645568 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 31 12:21:03.645568 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:21:03.645568 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:21:03.645568 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:21:03.645568 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:21:03.645568 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:21:03.645568 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:21:03.646505 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:21:03.646505 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:21:03.646505 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 31 12:21:03.646505 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 31 12:21:03.646505 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 31 12:21:03.646505 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 31 12:21:03.646505 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 31 12:21:03.646505 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 31 12:21:03.646505 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 31 12:21:03.646505 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:21:03.649036 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 305/595] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Oct 31 12:21:03 2022] GET /identity/v3/users/swift => generated 83 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:21:03.656490 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182b2d2e-947b-4975-bfe5-329f4d8f731e None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:03.686281 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70034ff-b059-4153-8170-70e893d45a25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:03.686906 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70034ff-b059-4153-8170-70e893d45a25 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:03.687203 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70034ff-b059-4153-8170-70e893d45a25 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:03.687485 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70034ff-b059-4153-8170-70e893d45a25 None admin] PATH_INFO: `/v3/services` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:03.688095 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70034ff-b059-4153-8170-70e893d45a25 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:03.689501 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70034ff-b059-4153-8170-70e893d45a25 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:03.699209 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 291/596] 10.222.0.15 () {56 vars in 1087 bytes} [Mon Oct 31 12:21:03 2022] GET /identity/v3/services => generated 772 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:21:03.783803 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182b2d2e-947b-4975-bfe5-329f4d8f731e None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:03.791892 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:03.821020 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b74375-f8e4-47bd-8190-9a82d587b614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:03.821635 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b74375-f8e4-47bd-8190-9a82d587b614 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:03.821909 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b74375-f8e4-47bd-8190-9a82d587b614 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:03.822367 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b74375-f8e4-47bd-8190-9a82d587b614 None admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:03.823217 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b74375-f8e4-47bd-8190-9a82d587b614 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:03.823534 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b74375-f8e4-47bd-8190-9a82d587b614 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:03.825069 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b74375-f8e4-47bd-8190-9a82d587b614 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:03.846764 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 306/597] 10.222.0.15 () {56 vars in 1135 bytes} [Mon Oct 31 12:21:03 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:21:03.852331 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70034ff-b059-4153-8170-70e893d45a25 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:03.975231 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70034ff-b059-4153-8170-70e893d45a25 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:04.015218 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524e000c-c629-455b-a0eb-f7a17f5c7379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:04.015836 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524e000c-c629-455b-a0eb-f7a17f5c7379 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:04.016122 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524e000c-c629-455b-a0eb-f7a17f5c7379 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:04.016393 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524e000c-c629-455b-a0eb-f7a17f5c7379 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:04.043956 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c9d3e5-2bdb-4dd7-acce-9eefdfd0e464 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:04.044318 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c9d3e5-2bdb-4dd7-acce-9eefdfd0e464 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:04.044509 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c9d3e5-2bdb-4dd7-acce-9eefdfd0e464 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:04.046194 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 307/598] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:21:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:21:04.052652 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8449b873-5db5-4067-99f2-b92ba4e13982 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:04.052942 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8449b873-5db5-4067-99f2-b92ba4e13982 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:04.053216 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8449b873-5db5-4067-99f2-b92ba4e13982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:04.059846 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:04.078071 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-8449b873-5db5-4067-99f2-b92ba4e13982 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:21:04.079433 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-8449b873-5db5-4067-99f2-b92ba4e13982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:21:04.100275 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524e000c-c629-455b-a0eb-f7a17f5c7379 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:04.101718 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524e000c-c629-455b-a0eb-f7a17f5c7379 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:04.171584 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 308/599] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:21:04 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 31 12:21:04.240312 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 292/600] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Oct 31 12:21:03 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 31 12:21:04.246618 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8449b873-5db5-4067-99f2-b92ba4e13982 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:04.393142 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:04.401128 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8449b873-5db5-4067-99f2-b92ba4e13982 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:04.451247 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bf7075-af19-4835-9296-6525cffa7bd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:04.451895 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bf7075-af19-4835-9296-6525cffa7bd7 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:04.452172 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bf7075-af19-4835-9296-6525cffa7bd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:04.452444 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bf7075-af19-4835-9296-6525cffa7bd7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:04.470741 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bf7075-af19-4835-9296-6525cffa7bd7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:04.472268 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bf7075-af19-4835-9296-6525cffa7bd7 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:04.492380 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-42bf7075-af19-4835-9296-6525cffa7bd7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:21:04.492380 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:04.492380 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:21:04.492380 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:21:04.492380 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:21:04.492380 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:21:04.492380 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:21:04.492380 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:21:04.492380 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:21:04.492380 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:21:04.492380 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:21:04.492380 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:21:04.492380 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 31 12:21:04.492380 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 31 12:21:04.492380 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 31 12:21:04.492380 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 31 12:21:04.493361 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:21:04.493361 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:21:04.493361 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:21:04.493361 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:21:04.493361 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:21:04.493361 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:21:04.493361 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:21:04.493361 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 31 12:21:04.493361 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:21:04.493361 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:21:04.493361 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:21:04.493361 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:21:04.493361 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:21:04.493361 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:21:04.493361 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:21:04.493361 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:21:04.493361 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 31 12:21:04.494764 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 31 12:21:04.494764 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:21:04.494764 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:21:04.494764 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 309/601] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Oct 31 12:21:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:21:04.500901 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524e000c-c629-455b-a0eb-f7a17f5c7379 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:04.526627 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bf7075-af19-4835-9296-6525cffa7bd7 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:04.608718 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:04.649738 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524e000c-c629-455b-a0eb-f7a17f5c7379 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:04.689921 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2235c487-05cb-44cf-bcae-3b21445edbf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:04.690590 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2235c487-05cb-44cf-bcae-3b21445edbf7 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:04.690875 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2235c487-05cb-44cf-bcae-3b21445edbf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:04.691145 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2235c487-05cb-44cf-bcae-3b21445edbf7 None admin] PATH_INFO: `/v3/domains` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:04.691815 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2235c487-05cb-44cf-bcae-3b21445edbf7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:04.692117 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2235c487-05cb-44cf-bcae-3b21445edbf7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:04.693603 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2235c487-05cb-44cf-bcae-3b21445edbf7 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:04.707028 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 293/602] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Oct 31 12:21:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:21:04.716128 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2235c487-05cb-44cf-bcae-3b21445edbf7 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:04.737861 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bf7075-af19-4835-9296-6525cffa7bd7 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:04.802347 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a393ccf1-a857-4cb2-a8e5-6019ce5b97f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:04.803287 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a393ccf1-a857-4cb2-a8e5-6019ce5b97f3 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:04.803705 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a393ccf1-a857-4cb2-a8e5-6019ce5b97f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:04.804132 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a393ccf1-a857-4cb2-a8e5-6019ce5b97f3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:04.822689 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a393ccf1-a857-4cb2-a8e5-6019ce5b97f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 31 12:21:04.828288 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-a393ccf1-a857-4cb2-a8e5-6019ce5b97f3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 31 12:21:04.828288 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:04.828288 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:21:04.828288 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:21:04.828288 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:21:04.828288 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:21:04.828288 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:21:04.828288 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:21:04.828288 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:21:04.828288 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:21:04.828288 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:21:04.828288 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:21:04.828288 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 31 12:21:04.828288 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 31 12:21:04.828288 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 31 12:21:04.828288 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 31 12:21:04.829407 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 31 12:21:04.829407 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 31 12:21:04.829407 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 31 12:21:04.829407 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 31 12:21:04.829407 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:21:04.829407 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:21:04.829407 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:21:04.829407 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:21:04.829407 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:21:04.829407 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:21:04.829407 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:21:04.829407 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 31 12:21:04.829407 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:21:04.829407 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:21:04.829407 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:21:04.829407 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:21:04.830416 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:21:04.830416 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:21:04.830416 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:21:04.830416 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:21:04.830416 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 31 12:21:04.830416 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 31 12:21:04.830416 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 31 12:21:04.830416 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 31 12:21:04.830416 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 31 12:21:04.830416 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 31 12:21:04.830416 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 31 12:21:04.830416 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:21:04.832642 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 310/603] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Oct 31 12:21:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 31 12:21:04.841770 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a393ccf1-a857-4cb2-a8e5-6019ce5b97f3 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:04.851484 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:04.915640 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2235c487-05cb-44cf-bcae-3b21445edbf7 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:04.954030 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:04.965401 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e8009b-0879-49f9-8bba-7d1c7eed805f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:04.966365 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e8009b-0879-49f9-8bba-7d1c7eed805f None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:04.966819 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e8009b-0879-49f9-8bba-7d1c7eed805f None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:04.967276 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e8009b-0879-49f9-8bba-7d1c7eed805f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:04.974971 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e8009b-0879-49f9-8bba-7d1c7eed805f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:04.976791 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e8009b-0879-49f9-8bba-7d1c7eed805f None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:04.990233 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-43e8009b-0879-49f9-8bba-7d1c7eed805f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 31 12:21:04.990233 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:04.990233 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:21:04.990233 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:21:04.990233 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:21:04.990233 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:21:04.990233 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:21:04.990233 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:21:04.990233 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:21:04.990233 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:21:04.990233 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:21:04.990233 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:21:04.990233 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 31 12:21:04.990233 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 31 12:21:04.990233 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 31 12:21:04.991412 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 31 12:21:04.991412 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:21:04.991412 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:21:04.991412 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:21:04.991412 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:21:04.991412 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:21:04.991412 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:21:04.991412 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:21:04.991412 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:21:04.991412 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:21:04.991412 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:21:04.991412 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:21:04.991412 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:21:04.991412 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:21:04.991412 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:21:04.991412 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:21:04.991412 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:21:04.992568 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 31 12:21:04.992568 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 31 12:21:04.992568 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 31 12:21:04.992568 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 31 12:21:04.992568 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 31 12:21:04.992568 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 31 12:21:04.992568 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 31 12:21:04.992568 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:21:04.995894 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 294/604] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Oct 31 12:21:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:21:05.001803 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e8009b-0879-49f9-8bba-7d1c7eed805f None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:05.036238 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a393ccf1-a857-4cb2-a8e5-6019ce5b97f3 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:05.111738 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5af75e1-f00e-42d0-a63b-de3ca8dd3de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:05.112737 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5af75e1-f00e-42d0-a63b-de3ca8dd3de9 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:05.113503 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5af75e1-f00e-42d0-a63b-de3ca8dd3de9 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:05.114200 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5af75e1-f00e-42d0-a63b-de3ca8dd3de9 None admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:05.115017 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5af75e1-f00e-42d0-a63b-de3ca8dd3de9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:05.115497 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5af75e1-f00e-42d0-a63b-de3ca8dd3de9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:05.116995 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5af75e1-f00e-42d0-a63b-de3ca8dd3de9 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:05.127946 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 311/605] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Oct 31 12:21:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:21:05.133788 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5494dec-912a-4568-8365-96543d11056b None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:05.133788 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5494dec-912a-4568-8365-96543d11056b None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:05.133788 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5494dec-912a-4568-8365-96543d11056b None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:05.135429 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 312/606] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:21:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:21:05.136371 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5494dec-912a-4568-8365-96543d11056b None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:05.167425 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e8009b-0879-49f9-8bba-7d1c7eed805f None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:05.204117 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce197371-cbfb-422e-83c3-20ce489b9229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:05.204802 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce197371-cbfb-422e-83c3-20ce489b9229 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:05.205077 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce197371-cbfb-422e-83c3-20ce489b9229 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:05.205346 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce197371-cbfb-422e-83c3-20ce489b9229 None admin] PATH_INFO: `/v3/projects` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:05.206078 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce197371-cbfb-422e-83c3-20ce489b9229 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:05.206385 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce197371-cbfb-422e-83c3-20ce489b9229 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:05.207829 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce197371-cbfb-422e-83c3-20ce489b9229 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:05.221067 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 295/607] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Oct 31 12:21:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:21:05.222327 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c5aebc-8b68-419f-9491-d0f81614158f None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:05.222612 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c5aebc-8b68-419f-9491-d0f81614158f None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:05.222937 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c5aebc-8b68-419f-9491-d0f81614158f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:05.242645 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-25c5aebc-8b68-419f-9491-d0f81614158f None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:21:05.242825 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:05.245224 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-25c5aebc-8b68-419f-9491-d0f81614158f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:21:05.286730 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5494dec-912a-4568-8365-96543d11056b None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:05.292307 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:05.325248 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 296/608] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:21:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 31 12:21:05.326438 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c5aebc-8b68-419f-9491-d0f81614158f None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:05.329307 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf885b9-c208-44b6-a8ff-6f466eaac103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:05.330200 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf885b9-c208-44b6-a8ff-6f466eaac103 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:05.330674 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf885b9-c208-44b6-a8ff-6f466eaac103 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:05.331220 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf885b9-c208-44b6-a8ff-6f466eaac103 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:05.344570 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf885b9-c208-44b6-a8ff-6f466eaac103 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:05.346760 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf885b9-c208-44b6-a8ff-6f466eaac103 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:05.362658 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-cdf885b9-c208-44b6-a8ff-6f466eaac103 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:21:05.362658 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:05.362658 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:21:05.362658 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:21:05.362658 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:21:05.362658 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:21:05.362658 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:21:05.362658 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:21:05.362658 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:21:05.362658 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:21:05.362658 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:21:05.362658 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:21:05.362658 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 31 12:21:05.362658 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 31 12:21:05.362658 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 31 12:21:05.362658 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 31 12:21:05.363654 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:21:05.363654 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:21:05.363654 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:21:05.363654 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:21:05.363654 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:21:05.363654 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:21:05.363654 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:21:05.363654 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 31 12:21:05.363654 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:21:05.363654 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:21:05.363654 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:21:05.363654 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:21:05.363654 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:21:05.363654 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:21:05.363654 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:21:05.363654 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:21:05.363654 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 31 12:21:05.364570 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 31 12:21:05.364570 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:21:05.364570 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:21:05.367149 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 313/609] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Oct 31 12:21:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:21:05.373561 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf885b9-c208-44b6-a8ff-6f466eaac103 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:05.471922 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c5aebc-8b68-419f-9491-d0f81614158f None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:05.493417 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf885b9-c208-44b6-a8ff-6f466eaac103 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:05.525580 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9fb496-edf9-4290-b767-1f5d6d459947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:05.526714 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9fb496-edf9-4290-b767-1f5d6d459947 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:05.527230 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9fb496-edf9-4290-b767-1f5d6d459947 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:05.527740 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9fb496-edf9-4290-b767-1f5d6d459947 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:05.529231 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9fb496-edf9-4290-b767-1f5d6d459947 None admin] RBAC: Adding query filter params (role.id=176c07b4fe3b4537bdfeb4be21b379c5, scope.project.id=2c32444cef584bce966b38203ce128b2, user.id=b2254a5786c249b4aa79244f7198ae65) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:05.529749 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9fb496-edf9-4290-b767-1f5d6d459947 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:05.531402 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9fb496-edf9-4290-b767-1f5d6d459947 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:05.538438 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 297/610] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Oct 31 12:21:05 2022] GET /identity/v3/role_assignments?user.id=b2254a5786c249b4aa79244f7198ae65&scope.project.id=2c32444cef584bce966b38203ce128b2&role.id=176c07b4fe3b4537bdfeb4be21b379c5 => generated 608 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:21:05.547544 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:05.559889 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900f63a8-ea51-4b25-913a-d13f03b29743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:05.560517 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900f63a8-ea51-4b25-913a-d13f03b29743 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:05.560797 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900f63a8-ea51-4b25-913a-d13f03b29743 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:05.561128 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900f63a8-ea51-4b25-913a-d13f03b29743 None admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:05.561793 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900f63a8-ea51-4b25-913a-d13f03b29743 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:05.562141 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900f63a8-ea51-4b25-913a-d13f03b29743 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:05.563582 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900f63a8-ea51-4b25-913a-d13f03b29743 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:05.580936 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 314/611] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Oct 31 12:21:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 31 12:21:05.586516 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9fb496-edf9-4290-b767-1f5d6d459947 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:05.632187 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:05.705016 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9fb496-edf9-4290-b767-1f5d6d459947 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:05.744941 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fbac3d-4ee2-41ca-af15-8f53bc5326e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:05.745632 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fbac3d-4ee2-41ca-af15-8f53bc5326e1 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:05.745912 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fbac3d-4ee2-41ca-af15-8f53bc5326e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:05.746184 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fbac3d-4ee2-41ca-af15-8f53bc5326e1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:05.755374 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fbac3d-4ee2-41ca-af15-8f53bc5326e1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:05.756863 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fbac3d-4ee2-41ca-af15-8f53bc5326e1 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:05.762792 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900f63a8-ea51-4b25-913a-d13f03b29743 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:05.774929 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-83fbac3d-4ee2-41ca-af15-8f53bc5326e1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 31 12:21:05.774929 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:05.774929 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:21:05.774929 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:21:05.774929 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:21:05.774929 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:21:05.774929 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:21:05.774929 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:21:05.774929 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:21:05.774929 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:21:05.774929 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:21:05.774929 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:21:05.774929 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 31 12:21:05.774929 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 31 12:21:05.774929 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 31 12:21:05.774929 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 31 12:21:05.776691 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:21:05.776691 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:21:05.776691 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 31 12:21:05.776691 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 31 12:21:05.776691 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 31 12:21:05.776691 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 31 12:21:05.776691 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:21:05.776691 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:21:05.776691 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:21:05.776691 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:21:05.776691 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:21:05.776691 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:21:05.776691 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:21:05.776691 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:21:05.776691 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:21:05.776691 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:21:05.776691 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:21:05.776691 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:21:05.777607 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:21:05.777607 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:21:05.777607 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 31 12:21:05.777607 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 31 12:21:05.777607 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 31 12:21:05.777607 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 31 12:21:05.777607 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 31 12:21:05.777607 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 31 12:21:05.777607 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 31 12:21:05.777607 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:21:05.780095 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 298/612] 10.222.0.15 () {56 vars in 1117 bytes} [Mon Oct 31 12:21:05 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:21:05.785346 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fbac3d-4ee2-41ca-af15-8f53bc5326e1 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:05.888968 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900f63a8-ea51-4b25-913a-d13f03b29743 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:05.929439 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e7bc6b-f7f3-42eb-b967-413f7ddab5d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:05.930390 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:05.931361 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e7bc6b-f7f3-42eb-b967-413f7ddab5d7 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:05.932611 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e7bc6b-f7f3-42eb-b967-413f7ddab5d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:05.933057 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e7bc6b-f7f3-42eb-b967-413f7ddab5d7 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:05.933826 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e7bc6b-f7f3-42eb-b967-413f7ddab5d7 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:05.935402 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e7bc6b-f7f3-42eb-b967-413f7ddab5d7 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:05.944059 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fbac3d-4ee2-41ca-af15-8f53bc5326e1 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:05.958076 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-09e7bc6b-f7f3-42eb-b967-413f7ddab5d7 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 31 12:21:05.958076 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:05.958076 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 31 12:21:05.958076 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 31 12:21:05.958076 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 31 12:21:05.958076 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 31 12:21:05.958076 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 31 12:21:05.958076 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 31 12:21:05.958076 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 31 12:21:05.958076 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:21:05.958076 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 31 12:21:05.958076 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:21:05.958076 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:05.958076 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:21:05.958076 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:21:05.958076 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:21:05.958076 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:21:05.959086 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:21:05.959086 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:21:05.959086 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:21:05.959086 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:21:05.959086 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:21:05.959086 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:21:05.959086 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 31 12:21:05.959086 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 31 12:21:05.959086 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 31 12:21:05.959086 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 31 12:21:05.959086 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:21:05.959086 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:21:05.959086 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:21:05.959086 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:21:05.959086 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:21:05.959086 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:21:05.959954 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:21:05.959954 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 31 12:21:05.959954 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:21:05.959954 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:21:05.959954 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:21:05.959954 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:21:05.959954 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:21:05.959954 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:21:05.959954 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:21:05.959954 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:21:05.959954 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 31 12:21:05.959954 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 31 12:21:05.959954 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 31 12:21:05.959954 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:21:05.964135 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 315/613] 10.222.0.15 () {56 vars in 1129 bytes} [Mon Oct 31 12:21:05 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:21:05.969668 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e7bc6b-f7f3-42eb-b967-413f7ddab5d7 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:06.007855 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5366f8ba-a939-4d8a-af03-e7ce54eef3d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:06.009057 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5366f8ba-a939-4d8a-af03-e7ce54eef3d0 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:06.009474 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5366f8ba-a939-4d8a-af03-e7ce54eef3d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:06.010148 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5366f8ba-a939-4d8a-af03-e7ce54eef3d0 None admin] PATH_INFO: `/v3/users` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:06.011298 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5366f8ba-a939-4d8a-af03-e7ce54eef3d0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:06.011745 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5366f8ba-a939-4d8a-af03-e7ce54eef3d0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:06.013305 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5366f8ba-a939-4d8a-af03-e7ce54eef3d0 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:06.042807 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 299/614] 10.222.0.15 () {56 vars in 1149 bytes} [Mon Oct 31 12:21:05 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 31 12:21:06.044506 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438832d0-bf4e-489e-a73b-0ccd746259b0 None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:06.045089 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438832d0-bf4e-489e-a73b-0ccd746259b0 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:06.046227 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438832d0-bf4e-489e-a73b-0ccd746259b0 None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:06.047783 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 300/615] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:21:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:21:06.048905 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438832d0-bf4e-489e-a73b-0ccd746259b0 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:06.061559 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:06.129381 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e7bc6b-f7f3-42eb-b967-413f7ddab5d7 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:06.171573 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f301d2-5007-442c-ae7b-baf8733958cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:06.172512 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f301d2-5007-442c-ae7b-baf8733958cb None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:06.172995 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f301d2-5007-442c-ae7b-baf8733958cb None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:06.173528 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f301d2-5007-442c-ae7b-baf8733958cb None admin] PATH_INFO: `/v3/services` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:06.174454 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f301d2-5007-442c-ae7b-baf8733958cb None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:06.174946 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f301d2-5007-442c-ae7b-baf8733958cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:06.176746 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f301d2-5007-442c-ae7b-baf8733958cb None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:06.182756 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 316/616] 10.222.0.15 () {56 vars in 1124 bytes} [Mon Oct 31 12:21:05 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:21:06.184485 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed49a78d-7c64-40e5-9193-5b3913b2f126 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:06.185003 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed49a78d-7c64-40e5-9193-5b3913b2f126 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:06.185514 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed49a78d-7c64-40e5-9193-5b3913b2f126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:06.196624 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438832d0-bf4e-489e-a73b-0ccd746259b0 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:06.206051 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-ed49a78d-7c64-40e5-9193-5b3913b2f126 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:21:06.207588 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-ed49a78d-7c64-40e5-9193-5b3913b2f126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:21:06.234785 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7642cb4-9322-4c32-951f-050c07d4bc88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:06.235429 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7642cb4-9322-4c32-951f-050c07d4bc88 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:06.235711 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7642cb4-9322-4c32-951f-050c07d4bc88 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:06.235983 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7642cb4-9322-4c32-951f-050c07d4bc88 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:06.263547 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:06.299263 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 317/617] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:21:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 31 12:21:06.300154 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed49a78d-7c64-40e5-9193-5b3913b2f126 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:06.317253 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7642cb4-9322-4c32-951f-050c07d4bc88 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:06.318961 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7642cb4-9322-4c32-951f-050c07d4bc88 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:06.382870 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:06.464098 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 301/618] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Oct 31 12:21:06 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 31 12:21:06.469831 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7642cb4-9322-4c32-951f-050c07d4bc88 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:06.540765 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed49a78d-7c64-40e5-9193-5b3913b2f126 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:06.575653 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:06.607585 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef5437b-896b-4b22-bcc3-9336f8f103e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:06.608667 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef5437b-896b-4b22-bcc3-9336f8f103e7 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:06.608944 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef5437b-896b-4b22-bcc3-9336f8f103e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:06.609220 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef5437b-896b-4b22-bcc3-9336f8f103e7 None admin] PATH_INFO: `/v3/services` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:06.609851 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef5437b-896b-4b22-bcc3-9336f8f103e7 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:06.610153 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef5437b-896b-4b22-bcc3-9336f8f103e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:06.612342 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef5437b-896b-4b22-bcc3-9336f8f103e7 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:06.619486 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 318/619] 10.222.0.15 () {56 vars in 1124 bytes} [Mon Oct 31 12:21:06 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:21:06.620692 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e436028-2900-49a7-9e80-f38a90f27d71 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:06.621036 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e436028-2900-49a7-9e80-f38a90f27d71 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:06.621303 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e436028-2900-49a7-9e80-f38a90f27d71 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:06.623392 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 319/620] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:21:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:21:06.626696 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e436028-2900-49a7-9e80-f38a90f27d71 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:06.635464 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7642cb4-9322-4c32-951f-050c07d4bc88 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:06.674625 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef4b59e-a604-44c2-be4e-68b48dfc4a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:06.675660 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef4b59e-a604-44c2-be4e-68b48dfc4a0f None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:06.675939 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef4b59e-a604-44c2-be4e-68b48dfc4a0f None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:06.676209 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef4b59e-a604-44c2-be4e-68b48dfc4a0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:06.687589 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef4b59e-a604-44c2-be4e-68b48dfc4a0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:06.689128 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef4b59e-a604-44c2-be4e-68b48dfc4a0f None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:06.704784 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-5ef4b59e-a604-44c2-be4e-68b48dfc4a0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:21:06.704784 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:06.704784 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:21:06.704784 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:21:06.704784 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:21:06.704784 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:21:06.704784 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:21:06.704784 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:21:06.704784 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:21:06.704784 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:21:06.704784 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:21:06.704784 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:21:06.704784 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 31 12:21:06.704784 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 31 12:21:06.704784 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 31 12:21:06.704784 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 31 12:21:06.705811 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:21:06.705811 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:21:06.705811 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:21:06.705811 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:21:06.705811 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:21:06.705811 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:21:06.705811 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:21:06.705811 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:21:06.705811 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:21:06.705811 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:21:06.705811 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:21:06.705811 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:21:06.705811 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:21:06.705811 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:21:06.705811 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:21:06.705811 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:21:06.705811 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 31 12:21:06.707327 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 31 12:21:06.707327 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:21:06.707327 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:21:06.707327 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 302/621] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Oct 31 12:21:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:21:06.707327 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8580c36-4093-48ed-a455-542a8f8b8c9a None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:06.707620 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8580c36-4093-48ed-a455-542a8f8b8c9a None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:06.707872 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8580c36-4093-48ed-a455-542a8f8b8c9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:06.727626 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-d8580c36-4093-48ed-a455-542a8f8b8c9a None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:21:06.728945 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-d8580c36-4093-48ed-a455-542a8f8b8c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:21:06.739868 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e436028-2900-49a7-9e80-f38a90f27d71 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:06.741507 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:06.777188 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f87d095-5672-4490-b46c-50b2a1a09fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:06.777820 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f87d095-5672-4490-b46c-50b2a1a09fb7 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:06.778355 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f87d095-5672-4490-b46c-50b2a1a09fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:06.778355 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f87d095-5672-4490-b46c-50b2a1a09fb7 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:06.778914 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f87d095-5672-4490-b46c-50b2a1a09fb7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:06.780301 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f87d095-5672-4490-b46c-50b2a1a09fb7 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:06.789053 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-5f87d095-5672-4490-b46c-50b2a1a09fb7 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 31 12:21:06.789053 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:06.789053 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 31 12:21:06.789053 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 31 12:21:06.789053 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 31 12:21:06.789053 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 31 12:21:06.789053 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 31 12:21:06.789053 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 31 12:21:06.789053 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 31 12:21:06.789053 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:21:06.789053 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 31 12:21:06.789053 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:21:06.789053 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:06.789053 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:21:06.789053 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:21:06.789053 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:21:06.789982 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:21:06.789982 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:21:06.789982 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:21:06.789982 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:21:06.789982 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:21:06.789982 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:21:06.789982 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:21:06.789982 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 31 12:21:06.789982 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 31 12:21:06.789982 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 31 12:21:06.789982 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 31 12:21:06.789982 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:21:06.789982 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:21:06.789982 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:21:06.789982 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:21:06.789982 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:21:06.789982 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:21:06.791010 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:21:06.791010 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 31 12:21:06.791010 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:21:06.791010 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:21:06.791010 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:21:06.791010 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:21:06.791010 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:21:06.791010 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:21:06.791010 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:21:06.791010 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:21:06.791010 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 31 12:21:06.791010 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 31 12:21:06.791010 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 31 12:21:06.791010 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:21:06.791010 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 320/622] 10.222.0.15 () {56 vars in 1132 bytes} [Mon Oct 31 12:21:06 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 31 12:21:06.791913 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f87d095-5672-4490-b46c-50b2a1a09fb7 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:06.811943 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 303/623] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:21:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 31 12:21:06.812614 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8580c36-4093-48ed-a455-542a8f8b8c9a None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:06.856849 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:06.912561 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f87d095-5672-4490-b46c-50b2a1a09fb7 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:06.923120 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8580c36-4093-48ed-a455-542a8f8b8c9a None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:06.951627 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2982d2ee-851c-4a5c-b121-8ce684230521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:06.952329 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2982d2ee-851c-4a5c-b121-8ce684230521 None admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:06.952609 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2982d2ee-851c-4a5c-b121-8ce684230521 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:06.952874 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2982d2ee-851c-4a5c-b121-8ce684230521 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:06.953524 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2982d2ee-851c-4a5c-b121-8ce684230521 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:06.954994 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2982d2ee-851c-4a5c-b121-8ce684230521 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:06.962348 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c1753c-c7c1-40d3-94d0-acac0b1bff41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:06.963603 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c1753c-c7c1-40d3-94d0-acac0b1bff41 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:06.964102 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c1753c-c7c1-40d3-94d0-acac0b1bff41 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:06.964686 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c1753c-c7c1-40d3-94d0-acac0b1bff41 None admin] PATH_INFO: `/v3/domains` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:06.965685 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c1753c-c7c1-40d3-94d0-acac0b1bff41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:06.966195 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c1753c-c7c1-40d3-94d0-acac0b1bff41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:06.968092 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c1753c-c7c1-40d3-94d0-acac0b1bff41 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:06.983075 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 321/624] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:21:06 2022] POST /identity/v3/endpoints => generated 346 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:21:06.984295 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2982d2ee-851c-4a5c-b121-8ce684230521 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:06.987096 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 304/625] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Oct 31 12:21:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:21:06.988108 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c1753c-c7c1-40d3-94d0-acac0b1bff41 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:07.031483 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:07.110690 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:07.116308 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c1753c-c7c1-40d3-94d0-acac0b1bff41 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:07.137623 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2982d2ee-851c-4a5c-b121-8ce684230521 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:07.155747 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd608e4a-e382-4dee-ab0d-076cdf00d569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:07.156482 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd608e4a-e382-4dee-ab0d-076cdf00d569 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:07.156846 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd608e4a-e382-4dee-ab0d-076cdf00d569 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:07.157183 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd608e4a-e382-4dee-ab0d-076cdf00d569 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:07.164907 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd608e4a-e382-4dee-ab0d-076cdf00d569 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:07.167153 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd608e4a-e382-4dee-ab0d-076cdf00d569 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:07.178407 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e23999-16a4-47a2-a07c-d2ff80d534b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:07.179330 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e23999-16a4-47a2-a07c-d2ff80d534b2 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:07.179809 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e23999-16a4-47a2-a07c-d2ff80d534b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:07.180250 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e23999-16a4-47a2-a07c-d2ff80d534b2 None admin] PATH_INFO: `/v3/services` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:07.180760 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-cd608e4a-e382-4dee-ab0d-076cdf00d569 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 31 12:21:07.180760 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:07.180760 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:21:07.180760 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:21:07.180760 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:21:07.180760 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:21:07.180760 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:21:07.180760 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:21:07.180760 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:21:07.180760 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:21:07.180760 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:21:07.180760 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:21:07.180760 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 31 12:21:07.180760 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 31 12:21:07.180760 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 31 12:21:07.182281 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 31 12:21:07.182281 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:21:07.182281 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:21:07.182281 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:21:07.182281 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:21:07.182281 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:21:07.182281 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:21:07.182281 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:21:07.182281 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:21:07.182281 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:21:07.182281 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:21:07.182281 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:21:07.182281 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:21:07.182281 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:21:07.182281 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:21:07.182281 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:21:07.182281 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:21:07.183176 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 31 12:21:07.183176 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 31 12:21:07.183176 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 31 12:21:07.183176 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 31 12:21:07.183176 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 31 12:21:07.183176 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 31 12:21:07.183176 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 31 12:21:07.183176 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:21:07.183176 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 305/626] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Oct 31 12:21:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:21:07.184506 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e23999-16a4-47a2-a07c-d2ff80d534b2 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:07.185033 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e23999-16a4-47a2-a07c-d2ff80d534b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:07.186665 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e23999-16a4-47a2-a07c-d2ff80d534b2 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:07.188835 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd608e4a-e382-4dee-ab0d-076cdf00d569 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:07.192786 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 322/627] 10.222.0.15 () {56 vars in 1126 bytes} [Mon Oct 31 12:21:06 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:21:07.198687 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e23999-16a4-47a2-a07c-d2ff80d534b2 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:07.305266 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:07.315043 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd608e4a-e382-4dee-ab0d-076cdf00d569 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:07.333142 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e23999-16a4-47a2-a07c-d2ff80d534b2 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:07.352626 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d5cfe9-b28f-4aca-8ad9-1a48292950ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:07.353662 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d5cfe9-b28f-4aca-8ad9-1a48292950ab None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:07.354511 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d5cfe9-b28f-4aca-8ad9-1a48292950ab None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:07.354634 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d5cfe9-b28f-4aca-8ad9-1a48292950ab None admin] PATH_INFO: `/v3/projects` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:07.355956 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d5cfe9-b28f-4aca-8ad9-1a48292950ab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:07.355956 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d5cfe9-b28f-4aca-8ad9-1a48292950ab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:07.357083 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d5cfe9-b28f-4aca-8ad9-1a48292950ab None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:07.372802 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 306/628] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Oct 31 12:21:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:21:07.373666 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d5cfe9-b28f-4aca-8ad9-1a48292950ab None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:07.380619 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e420734a-6ae5-4ad8-8da4-511f45534df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:07.381238 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e420734a-6ae5-4ad8-8da4-511f45534df1 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:07.381528 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e420734a-6ae5-4ad8-8da4-511f45534df1 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:07.381788 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e420734a-6ae5-4ad8-8da4-511f45534df1 None admin] PATH_INFO: `/v3/services` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:07.382475 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e420734a-6ae5-4ad8-8da4-511f45534df1 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:07.382786 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e420734a-6ae5-4ad8-8da4-511f45534df1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:07.384200 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e420734a-6ae5-4ad8-8da4-511f45534df1 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:07.390247 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 323/629] 10.222.0.15 () {56 vars in 1126 bytes} [Mon Oct 31 12:21:07 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:21:07.391059 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e420734a-6ae5-4ad8-8da4-511f45534df1 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:07.401104 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:07.506826 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e420734a-6ae5-4ad8-8da4-511f45534df1 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:07.552189 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177741a4-7225-4d68-b088-076a5e19df52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:07.552936 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177741a4-7225-4d68-b088-076a5e19df52 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:07.553221 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177741a4-7225-4d68-b088-076a5e19df52 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:07.553497 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177741a4-7225-4d68-b088-076a5e19df52 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:07.554227 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177741a4-7225-4d68-b088-076a5e19df52 None admin] RBAC: Adding query filter params (role.id=9674a2c5acc74dfeb93cfd67dfd890dd, scope.project.id=2c32444cef584bce966b38203ce128b2, user.id=bf4faacf7df248d28f32d7014dadcc07) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:07.554536 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177741a4-7225-4d68-b088-076a5e19df52 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:07.556192 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177741a4-7225-4d68-b088-076a5e19df52 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:07.564367 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 324/630] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Oct 31 12:21:07 2022] GET /identity/v3/role_assignments?user.id=bf4faacf7df248d28f32d7014dadcc07&scope.project.id=2c32444cef584bce966b38203ce128b2&role.id=9674a2c5acc74dfeb93cfd67dfd890dd => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:21:07.575109 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d5cfe9-b28f-4aca-8ad9-1a48292950ab None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:07.615124 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c388119e-29f0-4771-af91-2d0542acc458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:07.615814 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c388119e-29f0-4771-af91-2d0542acc458 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:07.616091 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c388119e-29f0-4771-af91-2d0542acc458 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:07.616359 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c388119e-29f0-4771-af91-2d0542acc458 None admin] PATH_INFO: `/v3/services/network` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:07.616947 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c388119e-29f0-4771-af91-2d0542acc458 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:07.618399 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c388119e-29f0-4771-af91-2d0542acc458 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:07.630423 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-c388119e-29f0-4771-af91-2d0542acc458 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 31 12:21:07.630423 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:07.630423 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 31 12:21:07.630423 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 31 12:21:07.630423 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 31 12:21:07.630423 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 31 12:21:07.630423 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 31 12:21:07.630423 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 31 12:21:07.630423 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 31 12:21:07.630423 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:21:07.630423 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 31 12:21:07.630423 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:21:07.630423 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:07.630423 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:21:07.630423 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:21:07.630423 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:21:07.630423 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:21:07.631526 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:21:07.631526 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:21:07.631526 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:21:07.631526 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:21:07.631526 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:21:07.631526 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:21:07.631526 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 31 12:21:07.631526 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 31 12:21:07.631526 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 31 12:21:07.631526 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 31 12:21:07.631526 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:21:07.631526 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:21:07.631526 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:21:07.631526 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:21:07.631526 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:21:07.631526 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:21:07.632353 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:21:07.632353 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:21:07.632353 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:21:07.632353 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:21:07.632353 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:21:07.632353 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:21:07.632353 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:21:07.632353 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:21:07.632353 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:21:07.632353 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:21:07.632353 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 31 12:21:07.632353 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 31 12:21:07.632353 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 31 12:21:07.632353 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:21:07.632353 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 307/631] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Oct 31 12:21:07 2022] GET /identity/v3/services/network => generated 88 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:21:07.639051 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177741a4-7225-4d68-b088-076a5e19df52 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:07.658978 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:07.769803 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177741a4-7225-4d68-b088-076a5e19df52 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:07.808226 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee95a6e8-1fa1-4db5-bacd-0074208e808c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:07.808773 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee95a6e8-1fa1-4db5-bacd-0074208e808c None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:07.809050 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee95a6e8-1fa1-4db5-bacd-0074208e808c None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:07.809319 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee95a6e8-1fa1-4db5-bacd-0074208e808c None admin] PATH_INFO: `/v3/services` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:07.809954 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee95a6e8-1fa1-4db5-bacd-0074208e808c None admin] RBAC: Adding query filter params (name=network) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:07.810254 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee95a6e8-1fa1-4db5-bacd-0074208e808c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:07.811641 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee95a6e8-1fa1-4db5-bacd-0074208e808c None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:07.819664 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 325/632] 10.222.0.15 () {56 vars in 1112 bytes} [Mon Oct 31 12:21:07 2022] GET /identity/v3/services?name=network => generated 124 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:21:07.825556 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c388119e-29f0-4771-af91-2d0542acc458 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:07.942410 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:07.969929 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c388119e-29f0-4771-af91-2d0542acc458 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:08.020966 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac092ad0-4765-4a2c-ada7-f797043112da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:08.021597 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac092ad0-4765-4a2c-ada7-f797043112da None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:08.021874 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac092ad0-4765-4a2c-ada7-f797043112da None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:08.022144 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac092ad0-4765-4a2c-ada7-f797043112da None admin] PATH_INFO: `/v3/services` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:08.022785 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac092ad0-4765-4a2c-ada7-f797043112da None admin] RBAC: Adding query filter params (type=network) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:08.023085 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac092ad0-4765-4a2c-ada7-f797043112da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:08.024490 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac092ad0-4765-4a2c-ada7-f797043112da None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:08.038705 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 308/633] 10.222.0.15 () {56 vars in 1112 bytes} [Mon Oct 31 12:21:07 2022] GET /identity/v3/services?type=network => generated 350 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:21:08.045198 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee95a6e8-1fa1-4db5-bacd-0074208e808c None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:08.142466 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8da3df-f4a6-433c-b956-933d8bfa4de2 None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:08.142972 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8da3df-f4a6-433c-b956-933d8bfa4de2 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:08.142972 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8da3df-f4a6-433c-b956-933d8bfa4de2 None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:08.144560 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 309/634] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:21:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 31 12:21:08.160619 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75a3857-d1dd-4e43-97ff-8be61f39a559 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:08.160924 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75a3857-d1dd-4e43-97ff-8be61f39a559 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:08.161163 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75a3857-d1dd-4e43-97ff-8be61f39a559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:08.175804 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:08.197762 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-e75a3857-d1dd-4e43-97ff-8be61f39a559 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:21:08.198957 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-e75a3857-d1dd-4e43-97ff-8be61f39a559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:21:08.222023 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee95a6e8-1fa1-4db5-bacd-0074208e808c None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:08.281507 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d97d43-e9fc-4893-8e63-709d985b9534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:08.282624 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d97d43-e9fc-4893-8e63-709d985b9534 None admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:08.284601 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 310/635] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:21:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 31 12:21:08.287480 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d97d43-e9fc-4893-8e63-709d985b9534 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:08.287869 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d97d43-e9fc-4893-8e63-709d985b9534 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:08.288667 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d97d43-e9fc-4893-8e63-709d985b9534 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:08.290186 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d97d43-e9fc-4893-8e63-709d985b9534 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:08.330099 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 326/636] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:21:08 2022] POST /identity/v3/endpoints => generated 337 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:21:08.585871 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75a3857-d1dd-4e43-97ff-8be61f39a559 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:08.668609 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:08.696125 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75a3857-d1dd-4e43-97ff-8be61f39a559 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:08.742110 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eade6ff5-438d-45ae-9b06-28a6c1ed0dac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:08.742748 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eade6ff5-438d-45ae-9b06-28a6c1ed0dac None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:08.743024 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eade6ff5-438d-45ae-9b06-28a6c1ed0dac None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:08.743326 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eade6ff5-438d-45ae-9b06-28a6c1ed0dac None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:08.744486 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eade6ff5-438d-45ae-9b06-28a6c1ed0dac None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:08.745910 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eade6ff5-438d-45ae-9b06-28a6c1ed0dac None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:08.754848 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-eade6ff5-438d-45ae-9b06-28a6c1ed0dac None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 31 12:21:08.754848 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:08.754848 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 31 12:21:08.754848 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 31 12:21:08.754848 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 31 12:21:08.754848 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 31 12:21:08.754848 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 31 12:21:08.754848 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 31 12:21:08.754848 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 31 12:21:08.754848 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:21:08.754848 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 31 12:21:08.754848 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:21:08.754848 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:08.754848 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:21:08.754848 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:21:08.754848 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:21:08.754848 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:21:08.755861 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:21:08.755861 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:21:08.755861 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:21:08.755861 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:21:08.755861 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:21:08.755861 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:21:08.755861 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 31 12:21:08.755861 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 31 12:21:08.755861 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 31 12:21:08.755861 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 31 12:21:08.755861 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:21:08.755861 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:21:08.755861 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:21:08.755861 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:21:08.755861 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:21:08.755861 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:21:08.756695 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:21:08.756695 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:21:08.756695 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:21:08.756695 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:21:08.756695 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:21:08.756695 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:21:08.756695 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:21:08.756695 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:21:08.756695 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:21:08.756695 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:21:08.756695 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 31 12:21:08.756695 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 31 12:21:08.756695 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 31 12:21:08.756695 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:21:08.761422 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 311/637] 10.222.0.15 () {56 vars in 1126 bytes} [Mon Oct 31 12:21:08 2022] GET /identity/v3/services/object-store => generated 93 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:21:08.762776 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d97d43-e9fc-4893-8e63-709d985b9534 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:08.874870 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d97d43-e9fc-4893-8e63-709d985b9534 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:08.904769 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:08.911432 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0914c793-a153-4939-8d04-6436016f3456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:08.913225 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0914c793-a153-4939-8d04-6436016f3456 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:08.913530 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0914c793-a153-4939-8d04-6436016f3456 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:08.913898 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0914c793-a153-4939-8d04-6436016f3456 None admin] PATH_INFO: `/v3/services` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:08.914634 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0914c793-a153-4939-8d04-6436016f3456 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:08.914956 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0914c793-a153-4939-8d04-6436016f3456 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:08.917497 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0914c793-a153-4939-8d04-6436016f3456 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:08.922189 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 327/638] 10.222.0.15 () {56 vars in 1122 bytes} [Mon Oct 31 12:21:08 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:21:08.927998 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eade6ff5-438d-45ae-9b06-28a6c1ed0dac None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:09.049776 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eade6ff5-438d-45ae-9b06-28a6c1ed0dac None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:09.091715 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9674b410-2c91-4ed0-a39b-2c799af35f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:09.092329 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9674b410-2c91-4ed0-a39b-2c799af35f19 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:09.092608 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9674b410-2c91-4ed0-a39b-2c799af35f19 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:09.092874 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9674b410-2c91-4ed0-a39b-2c799af35f19 None admin] PATH_INFO: `/v3/services` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:09.093499 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9674b410-2c91-4ed0-a39b-2c799af35f19 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:09.093796 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9674b410-2c91-4ed0-a39b-2c799af35f19 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:09.095145 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9674b410-2c91-4ed0-a39b-2c799af35f19 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:09.097282 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:09.107360 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 312/639] 10.222.0.15 () {56 vars in 1122 bytes} [Mon Oct 31 12:21:08 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:21:09.635399 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599e4ed2-391c-4480-9554-f1aedfb0350e None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:09.635932 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599e4ed2-391c-4480-9554-f1aedfb0350e None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:09.635932 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599e4ed2-391c-4480-9554-f1aedfb0350e None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:09.637528 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 328/640] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:21:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:21:09.642859 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c0382d-abf5-4f88-8878-df80b1bb6693 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:09.643975 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c0382d-abf5-4f88-8878-df80b1bb6693 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:09.644450 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c0382d-abf5-4f88-8878-df80b1bb6693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:09.663620 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-58c0382d-abf5-4f88-8878-df80b1bb6693 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:21:09.665122 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-58c0382d-abf5-4f88-8878-df80b1bb6693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:21:09.734344 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 313/641] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:21:09 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 31 12:21:10.005450 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599e4ed2-391c-4480-9554-f1aedfb0350e None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:10.078220 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b17798f-9014-4498-a153-ed99a13cc570 None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:10.078878 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b17798f-9014-4498-a153-ed99a13cc570 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:10.079401 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b17798f-9014-4498-a153-ed99a13cc570 None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:10.081394 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 314/642] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:21:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 31 12:21:10.086817 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dab4a7-c402-447d-995a-7409eae9f659 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:10.087754 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dab4a7-c402-447d-995a-7409eae9f659 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:10.088534 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dab4a7-c402-447d-995a-7409eae9f659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:10.108663 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-10dab4a7-c402-447d-995a-7409eae9f659 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:21:10.110173 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-10dab4a7-c402-447d-995a-7409eae9f659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:21:10.122188 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599e4ed2-391c-4480-9554-f1aedfb0350e None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:10.157474 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c415ef0-df98-4ccf-a0fb-693317339230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:10.158043 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c415ef0-df98-4ccf-a0fb-693317339230 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:10.158318 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c415ef0-df98-4ccf-a0fb-693317339230 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:10.158585 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c415ef0-df98-4ccf-a0fb-693317339230 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:10.159202 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c415ef0-df98-4ccf-a0fb-693317339230 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:10.160533 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c415ef0-df98-4ccf-a0fb-693317339230 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:10.170617 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-8c415ef0-df98-4ccf-a0fb-693317339230 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 31 12:21:10.170617 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:10.170617 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 31 12:21:10.170617 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 31 12:21:10.170617 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 31 12:21:10.170617 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 31 12:21:10.170617 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 31 12:21:10.170617 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 31 12:21:10.170617 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 31 12:21:10.170617 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:21:10.170617 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 31 12:21:10.170617 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:21:10.170617 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:10.170617 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:21:10.170617 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:21:10.170617 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:21:10.170617 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:21:10.171638 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:21:10.171638 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:21:10.171638 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:21:10.171638 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:21:10.171638 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:21:10.171638 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:21:10.171638 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 31 12:21:10.171638 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 31 12:21:10.171638 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 31 12:21:10.171638 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 31 12:21:10.171638 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:21:10.171638 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:21:10.171638 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:21:10.171638 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:21:10.171638 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:21:10.171638 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:21:10.172486 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:21:10.172486 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 31 12:21:10.172486 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:21:10.172486 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:21:10.172486 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:21:10.172486 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:21:10.172486 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:21:10.172486 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:21:10.172486 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:21:10.172486 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:21:10.172486 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 31 12:21:10.172486 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 31 12:21:10.172486 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 31 12:21:10.172486 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:21:10.172486 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 329/643] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Oct 31 12:21:10 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:21:10.177398 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c415ef0-df98-4ccf-a0fb-693317339230 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:10.195463 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 315/644] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:21:10 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 31 12:21:10.196860 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ab6a1d-c228-4639-866c-f26f0b169f96 None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:10.197663 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ab6a1d-c228-4639-866c-f26f0b169f96 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:10.197935 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ab6a1d-c228-4639-866c-f26f0b169f96 None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:10.199665 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 316/645] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:21:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 31 12:21:10.204113 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65612d1-b206-42ad-a5f9-53628df86452 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:10.204561 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65612d1-b206-42ad-a5f9-53628df86452 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:10.204971 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65612d1-b206-42ad-a5f9-53628df86452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:10.224546 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-d65612d1-b206-42ad-a5f9-53628df86452 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:21:10.226032 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-d65612d1-b206-42ad-a5f9-53628df86452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:21:10.250723 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:10.291610 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:10.298153 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c415ef0-df98-4ccf-a0fb-693317339230 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:10.304473 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 317/646] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:21:10 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 31 12:21:10.335560 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdef7e52-7c7f-4d18-9e96-19eb3ee477bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:10.336170 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdef7e52-7c7f-4d18-9e96-19eb3ee477bf None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:10.336445 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdef7e52-7c7f-4d18-9e96-19eb3ee477bf None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:10.336713 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdef7e52-7c7f-4d18-9e96-19eb3ee477bf None admin] PATH_INFO: `/v3/services` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:10.337339 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdef7e52-7c7f-4d18-9e96-19eb3ee477bf None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:10.337710 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdef7e52-7c7f-4d18-9e96-19eb3ee477bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:10.339241 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdef7e52-7c7f-4d18-9e96-19eb3ee477bf None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:10.344099 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 330/647] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Oct 31 12:21:10 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 31 12:21:10.349372 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65612d1-b206-42ad-a5f9-53628df86452 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:10.453263 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65612d1-b206-42ad-a5f9-53628df86452 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:10.492190 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca4a571-b354-438e-8b70-89a0ced4470a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:10.492790 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca4a571-b354-438e-8b70-89a0ced4470a None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:10.493065 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca4a571-b354-438e-8b70-89a0ced4470a None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:10.493330 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca4a571-b354-438e-8b70-89a0ced4470a None admin] PATH_INFO: `/v3/services` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:10.494523 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca4a571-b354-438e-8b70-89a0ced4470a None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:10.494802 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca4a571-b354-438e-8b70-89a0ced4470a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:10.496180 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca4a571-b354-438e-8b70-89a0ced4470a None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:10.498933 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdef7e52-7c7f-4d18-9e96-19eb3ee477bf None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:10.501193 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 318/648] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Oct 31 12:21:10 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:21:10.584648 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca4a571-b354-438e-8b70-89a0ced4470a None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:10.617914 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdef7e52-7c7f-4d18-9e96-19eb3ee477bf None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:10.644902 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:10.646305 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:10.663562 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4d7f97-c6e0-4d3f-9931-e52032b82084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:10.664563 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4d7f97-c6e0-4d3f-9931-e52032b82084 None admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:10.665015 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4d7f97-c6e0-4d3f-9931-e52032b82084 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:10.665432 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4d7f97-c6e0-4d3f-9931-e52032b82084 None admin] PATH_INFO: `/v3/services` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:10.666989 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4d7f97-c6e0-4d3f-9931-e52032b82084 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:10.668676 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4d7f97-c6e0-4d3f-9931-e52032b82084 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:10.693073 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 331/649] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Oct 31 12:21:10 2022] POST /identity/v3/services => generated 269 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 31 12:21:10.712147 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca4a571-b354-438e-8b70-89a0ced4470a None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:10.764431 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393562a0-6b74-4e55-b3df-381942614f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:10.765080 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393562a0-6b74-4e55-b3df-381942614f9e None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:10.765355 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393562a0-6b74-4e55-b3df-381942614f9e None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:10.765623 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393562a0-6b74-4e55-b3df-381942614f9e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:10.779458 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393562a0-6b74-4e55-b3df-381942614f9e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 31 12:21:10.784464 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-393562a0-6b74-4e55-b3df-381942614f9e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 31 12:21:10.784464 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:10.784464 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:21:10.784464 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:21:10.784464 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:21:10.784464 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:21:10.784464 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:21:10.784464 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:21:10.784464 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:21:10.784464 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:21:10.784464 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:21:10.784464 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:21:10.784464 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 31 12:21:10.784464 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 31 12:21:10.784464 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 31 12:21:10.784464 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 31 12:21:10.785453 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 31 12:21:10.785453 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 31 12:21:10.785453 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 31 12:21:10.785453 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 31 12:21:10.785453 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:21:10.785453 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:21:10.785453 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:21:10.785453 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:21:10.785453 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:21:10.785453 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:21:10.785453 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:21:10.785453 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:21:10.785453 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:21:10.785453 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:21:10.785453 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:21:10.785453 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:21:10.786299 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:21:10.786299 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:21:10.786299 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:21:10.786299 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:21:10.786299 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 31 12:21:10.786299 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 31 12:21:10.786299 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 31 12:21:10.786299 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 31 12:21:10.786299 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 31 12:21:10.786299 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 31 12:21:10.786299 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 31 12:21:10.786299 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:21:10.786299 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 319/650] 10.222.0.15 () {56 vars in 1120 bytes} [Mon Oct 31 12:21:10 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 31 12:21:10.794921 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4d7f97-c6e0-4d3f-9931-e52032b82084 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:10.915571 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4d7f97-c6e0-4d3f-9931-e52032b82084 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:10.951386 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e14991-ae72-4bbd-9b53-4798ded7cb71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:10.951995 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e14991-ae72-4bbd-9b53-4798ded7cb71 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:10.952268 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e14991-ae72-4bbd-9b53-4798ded7cb71 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:10.952535 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e14991-ae72-4bbd-9b53-4798ded7cb71 None admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:10.953198 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e14991-ae72-4bbd-9b53-4798ded7cb71 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:10.953496 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e14991-ae72-4bbd-9b53-4798ded7cb71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:10.954842 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e14991-ae72-4bbd-9b53-4798ded7cb71 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:10.963768 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 332/651] 10.222.0.15 () {56 vars in 1115 bytes} [Mon Oct 31 12:21:10 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:21:10.972262 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393562a0-6b74-4e55-b3df-381942614f9e None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:11.086117 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393562a0-6b74-4e55-b3df-381942614f9e None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:11.098902 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:11.127667 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b25b91-97d5-4a32-9305-090b10570b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:11.128304 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b25b91-97d5-4a32-9305-090b10570b74 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:11.128584 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b25b91-97d5-4a32-9305-090b10570b74 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:11.128857 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b25b91-97d5-4a32-9305-090b10570b74 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:11.140569 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b25b91-97d5-4a32-9305-090b10570b74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:11.142019 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b25b91-97d5-4a32-9305-090b10570b74 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:11.157879 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-f9b25b91-97d5-4a32-9305-090b10570b74 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:21:11.157879 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:11.157879 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:21:11.157879 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:21:11.157879 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:21:11.157879 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:21:11.157879 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:21:11.157879 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:21:11.157879 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:21:11.157879 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:21:11.157879 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:21:11.157879 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:21:11.157879 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 31 12:21:11.157879 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 31 12:21:11.157879 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 31 12:21:11.157879 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 31 12:21:11.158828 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:21:11.158828 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:21:11.158828 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:21:11.158828 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:21:11.158828 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:21:11.158828 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:21:11.158828 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:21:11.158828 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:21:11.158828 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:21:11.158828 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:21:11.158828 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:21:11.158828 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:21:11.158828 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:21:11.158828 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:21:11.158828 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:21:11.158828 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:21:11.158828 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 31 12:21:11.159746 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 31 12:21:11.159746 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:21:11.159746 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:21:11.159746 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 320/652] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Oct 31 12:21:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:21:11.165063 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e14991-ae72-4bbd-9b53-4798ded7cb71 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:11.215745 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:11.275607 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e14991-ae72-4bbd-9b53-4798ded7cb71 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:11.311090 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d8ba24-580f-4ab4-8c34-2027fc0118d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:11.311779 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d8ba24-580f-4ab4-8c34-2027fc0118d7 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:11.312055 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d8ba24-580f-4ab4-8c34-2027fc0118d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:11.312323 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d8ba24-580f-4ab4-8c34-2027fc0118d7 None admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:11.313005 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d8ba24-580f-4ab4-8c34-2027fc0118d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:11.313310 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d8ba24-580f-4ab4-8c34-2027fc0118d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:11.314660 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d8ba24-580f-4ab4-8c34-2027fc0118d7 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:11.327113 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 333/653] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Oct 31 12:21:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:21:11.332697 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b25b91-97d5-4a32-9305-090b10570b74 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:11.373306 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e4d13e-6085-447e-b13e-2a6fa462d0ed None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:11.373649 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e4d13e-6085-447e-b13e-2a6fa462d0ed None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:11.373851 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e4d13e-6085-447e-b13e-2a6fa462d0ed None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:11.375333 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 334/654] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:21:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 31 12:21:11.380490 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22e8cb2-c700-48d1-bc5f-68592d17bf0e None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:11.380771 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22e8cb2-c700-48d1-bc5f-68592d17bf0e None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:11.381042 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22e8cb2-c700-48d1-bc5f-68592d17bf0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:11.405214 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-f22e8cb2-c700-48d1-bc5f-68592d17bf0e None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:21:11.407366 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-f22e8cb2-c700-48d1-bc5f-68592d17bf0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:21:11.461309 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b25b91-97d5-4a32-9305-090b10570b74 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:11.493048 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 335/655] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:21:11 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 31 12:21:11.506093 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a26c8d-73c0-4190-87f2-7ecaff9801b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:11.506953 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a26c8d-73c0-4190-87f2-7ecaff9801b8 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:11.507445 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a26c8d-73c0-4190-87f2-7ecaff9801b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:11.508001 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a26c8d-73c0-4190-87f2-7ecaff9801b8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:11.517822 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a26c8d-73c0-4190-87f2-7ecaff9801b8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:11.519781 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a26c8d-73c0-4190-87f2-7ecaff9801b8 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:11.532155 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-c5a26c8d-73c0-4190-87f2-7ecaff9801b8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 31 12:21:11.532155 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:11.532155 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:21:11.532155 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:21:11.532155 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:21:11.532155 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:21:11.532155 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:21:11.532155 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:21:11.532155 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:21:11.532155 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:21:11.532155 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:21:11.532155 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:21:11.532155 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 31 12:21:11.532155 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 31 12:21:11.532155 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 31 12:21:11.532155 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 31 12:21:11.533345 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:21:11.533345 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:21:11.533345 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 31 12:21:11.533345 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 31 12:21:11.533345 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 31 12:21:11.533345 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 31 12:21:11.533345 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:21:11.533345 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:21:11.533345 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:21:11.533345 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:21:11.533345 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:21:11.533345 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:21:11.533345 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:21:11.533345 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:21:11.533345 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:21:11.533345 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:21:11.533345 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:21:11.533345 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:21:11.534468 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:21:11.534468 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:21:11.534468 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 31 12:21:11.534468 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 31 12:21:11.534468 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 31 12:21:11.534468 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 31 12:21:11.534468 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 31 12:21:11.534468 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 31 12:21:11.534468 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 31 12:21:11.534468 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:21:11.536627 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 321/656] 10.222.0.15 () {56 vars in 1117 bytes} [Mon Oct 31 12:21:11 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:21:11.544087 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22e8cb2-c700-48d1-bc5f-68592d17bf0e None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:11.581765 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:11.651351 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22e8cb2-c700-48d1-bc5f-68592d17bf0e None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:11.686743 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d5a757-33ed-4881-9f69-f0f2c80add7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:11.687872 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d5a757-33ed-4881-9f69-f0f2c80add7d None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:11.688343 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d5a757-33ed-4881-9f69-f0f2c80add7d None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:11.688926 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d5a757-33ed-4881-9f69-f0f2c80add7d None admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:11.689856 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d5a757-33ed-4881-9f69-f0f2c80add7d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:11.690334 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d5a757-33ed-4881-9f69-f0f2c80add7d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:11.692028 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d5a757-33ed-4881-9f69-f0f2c80add7d None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:11.714108 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 336/657] 10.222.0.15 () {56 vars in 1149 bytes} [Mon Oct 31 12:21:11 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 31 12:21:11.720252 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a26c8d-73c0-4190-87f2-7ecaff9801b8 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:11.747312 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:11.765426 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d5a757-33ed-4881-9f69-f0f2c80add7d None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:11.846542 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a26c8d-73c0-4190-87f2-7ecaff9801b8 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:11.874424 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:11.886742 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d5a757-33ed-4881-9f69-f0f2c80add7d None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:11.894131 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928bd934-ee92-4974-82c3-4115d604235f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:11.894842 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928bd934-ee92-4974-82c3-4115d604235f None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:11.895214 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928bd934-ee92-4974-82c3-4115d604235f None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:11.895565 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928bd934-ee92-4974-82c3-4115d604235f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:11.910874 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928bd934-ee92-4974-82c3-4115d604235f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:11.913016 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928bd934-ee92-4974-82c3-4115d604235f None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:11.928589 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a70749-fe7f-425b-a900-9663ba9ef591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:11.929504 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a70749-fe7f-425b-a900-9663ba9ef591 None admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:11.929957 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a70749-fe7f-425b-a900-9663ba9ef591 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:11.930400 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a70749-fe7f-425b-a900-9663ba9ef591 None admin] PATH_INFO: `/v3/services` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:11.931903 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a70749-fe7f-425b-a900-9663ba9ef591 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:11.932651 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-928bd934-ee92-4974-82c3-4115d604235f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:21:11.932651 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:11.932651 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:21:11.932651 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:21:11.932651 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:21:11.932651 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:21:11.932651 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:21:11.932651 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:21:11.932651 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:21:11.932651 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:21:11.932651 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:21:11.932651 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:21:11.932651 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 31 12:21:11.932651 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 31 12:21:11.932651 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 31 12:21:11.932651 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 31 12:21:11.933586 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:21:11.933586 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:21:11.933586 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:21:11.933586 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:21:11.933586 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:21:11.933586 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:21:11.933586 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:21:11.933586 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:21:11.933586 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:21:11.933586 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:21:11.933586 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:21:11.933586 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:21:11.933586 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:21:11.933586 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:21:11.933586 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:21:11.933586 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:21:11.933586 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 31 12:21:11.934606 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 31 12:21:11.934606 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:21:11.934606 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:21:11.934606 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 322/658] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Oct 31 12:21:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:21:11.936253 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a70749-fe7f-425b-a900-9663ba9ef591 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:11.939130 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928bd934-ee92-4974-82c3-4115d604235f None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:11.953963 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 337/659] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Oct 31 12:21:11 2022] POST /identity/v3/services => generated 241 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 31 12:21:12.039364 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:12.069582 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928bd934-ee92-4974-82c3-4115d604235f None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:12.113060 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3404a5-8e91-4921-bfe1-98a31970f376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:12.113758 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3404a5-8e91-4921-bfe1-98a31970f376 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:12.114034 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3404a5-8e91-4921-bfe1-98a31970f376 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:12.114300 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3404a5-8e91-4921-bfe1-98a31970f376 None admin] PATH_INFO: `/v3/domains` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:12.114936 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3404a5-8e91-4921-bfe1-98a31970f376 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:12.115265 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3404a5-8e91-4921-bfe1-98a31970f376 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:12.116712 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3404a5-8e91-4921-bfe1-98a31970f376 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:12.130780 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 323/660] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Oct 31 12:21:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:21:12.140400 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a70749-fe7f-425b-a900-9663ba9ef591 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:12.247091 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a70749-fe7f-425b-a900-9663ba9ef591 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:12.281995 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb167e2a-04bc-4df5-9c8a-3c13a359400d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:12.282552 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb167e2a-04bc-4df5-9c8a-3c13a359400d None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:12.282833 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb167e2a-04bc-4df5-9c8a-3c13a359400d None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:12.283103 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb167e2a-04bc-4df5-9c8a-3c13a359400d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:12.289838 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb167e2a-04bc-4df5-9c8a-3c13a359400d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:12.291870 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb167e2a-04bc-4df5-9c8a-3c13a359400d None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:12.303196 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-fb167e2a-04bc-4df5-9c8a-3c13a359400d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 31 12:21:12.303196 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:12.303196 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:21:12.303196 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:21:12.303196 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:21:12.303196 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:21:12.303196 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:21:12.303196 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:21:12.303196 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:21:12.303196 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:21:12.303196 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:21:12.303196 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:21:12.303196 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 31 12:21:12.303196 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 31 12:21:12.303196 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 31 12:21:12.304083 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 31 12:21:12.304083 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:21:12.304083 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:21:12.304083 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:21:12.304083 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:21:12.304083 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:21:12.304083 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:21:12.304083 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:21:12.304083 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 31 12:21:12.304083 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:21:12.304083 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:21:12.304083 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:21:12.304083 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:21:12.304083 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:21:12.304083 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:21:12.304083 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:21:12.304083 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:21:12.304960 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 31 12:21:12.304960 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 31 12:21:12.304960 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 31 12:21:12.304960 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 31 12:21:12.304960 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 31 12:21:12.304960 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 31 12:21:12.304960 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 31 12:21:12.304960 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:21:12.304960 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 338/661] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Oct 31 12:21:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:21:12.312185 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3404a5-8e91-4921-bfe1-98a31970f376 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:12.421426 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3404a5-8e91-4921-bfe1-98a31970f376 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:12.451964 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:12.458079 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67274ba0-8da9-4a52-a57c-0df73d59ac28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:12.458984 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67274ba0-8da9-4a52-a57c-0df73d59ac28 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:12.459658 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67274ba0-8da9-4a52-a57c-0df73d59ac28 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:12.460174 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67274ba0-8da9-4a52-a57c-0df73d59ac28 None admin] PATH_INFO: `/v3/projects` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:12.461207 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67274ba0-8da9-4a52-a57c-0df73d59ac28 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:12.461507 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67274ba0-8da9-4a52-a57c-0df73d59ac28 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:12.463382 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67274ba0-8da9-4a52-a57c-0df73d59ac28 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:12.475994 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 324/662] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Oct 31 12:21:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:21:12.481749 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb167e2a-04bc-4df5-9c8a-3c13a359400d None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:12.498146 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:12.599175 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb167e2a-04bc-4df5-9c8a-3c13a359400d None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:12.636028 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badc3329-359f-41ce-bb9c-2cb0b5573981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:12.637086 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badc3329-359f-41ce-bb9c-2cb0b5573981 None admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:12.637577 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badc3329-359f-41ce-bb9c-2cb0b5573981 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:12.638047 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badc3329-359f-41ce-bb9c-2cb0b5573981 None admin] PATH_INFO: `/v3/projects/2c32444cef584bce966b38203ce128b2/users/bf4faacf7df248d28f32d7014dadcc07/roles/92746653d6b94cf5a4e22d9efd5d4e4f` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:12.640476 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badc3329-359f-41ce-bb9c-2cb0b5573981 None admin] RBAC: Authorizing `identity:create_grant(project_id=2c32444cef584bce966b38203ce128b2, user_id=bf4faacf7df248d28f32d7014dadcc07, role_id=92746653d6b94cf5a4e22d9efd5d4e4f)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:12.642112 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badc3329-359f-41ce-bb9c-2cb0b5573981 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:12.660427 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 339/663] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Oct 31 12:21:12 2022] PUT /identity/v3/projects/2c32444cef584bce966b38203ce128b2/users/bf4faacf7df248d28f32d7014dadcc07/roles/92746653d6b94cf5a4e22d9efd5d4e4f => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:21:12.850840 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e455d-7712-4012-b186-99927ec17917 None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:12.851395 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e455d-7712-4012-b186-99927ec17917 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:12.851395 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e455d-7712-4012-b186-99927ec17917 None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:12.852928 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 325/664] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:21:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:21:12.858491 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8229a25f-3054-4a47-a637-aa5d08b588b4 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:12.858776 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8229a25f-3054-4a47-a637-aa5d08b588b4 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:12.859045 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8229a25f-3054-4a47-a637-aa5d08b588b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:12.877721 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-8229a25f-3054-4a47-a637-aa5d08b588b4 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:21:12.878884 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-8229a25f-3054-4a47-a637-aa5d08b588b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:21:12.949442 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 340/665] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:21:12 2022] POST /identity/v3/auth/tokens => generated 1373 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 31 12:21:13.081124 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe232b6-b940-4061-9420-453582e92ca4 None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:13.081444 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe232b6-b940-4061-9420-453582e92ca4 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:13.081668 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe232b6-b940-4061-9420-453582e92ca4 None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:13.083308 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 326/666] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:21:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:21:13.088560 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fa9826-31f0-4483-b365-55b71e7071a7 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:13.088878 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fa9826-31f0-4483-b365-55b71e7071a7 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:13.089107 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fa9826-31f0-4483-b365-55b71e7071a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:13.095431 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:13.108505 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-b9fa9826-31f0-4483-b365-55b71e7071a7 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:21:13.109707 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-b9fa9826-31f0-4483-b365-55b71e7071a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:21:13.178800 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 341/667] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:21:13 2022] POST /identity/v3/auth/tokens => generated 1373 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 31 12:21:13.230886 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe232b6-b940-4061-9420-453582e92ca4 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:13.334069 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe232b6-b940-4061-9420-453582e92ca4 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:13.370007 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65c2174-c2fd-431d-8399-3896090b98be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:13.370946 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65c2174-c2fd-431d-8399-3896090b98be None admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:13.371464 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65c2174-c2fd-431d-8399-3896090b98be None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:13.371942 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65c2174-c2fd-431d-8399-3896090b98be None admin] PATH_INFO: `/v3/services` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:13.372788 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65c2174-c2fd-431d-8399-3896090b98be None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:13.374315 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65c2174-c2fd-431d-8399-3896090b98be None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:13.392967 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 327/668] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Oct 31 12:21:13 2022] POST /identity/v3/services => generated 251 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 31 12:21:13.460854 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fa9826-31f0-4483-b365-55b71e7071a7 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:13.573577 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fa9826-31f0-4483-b365-55b71e7071a7 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:13.606176 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:13.618306 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cca77f0-b6ab-4d1d-85f2-7ee1c6e01247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:13.619041 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cca77f0-b6ab-4d1d-85f2-7ee1c6e01247 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:13.620483 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cca77f0-b6ab-4d1d-85f2-7ee1c6e01247 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:13.620483 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cca77f0-b6ab-4d1d-85f2-7ee1c6e01247 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:13.620483 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cca77f0-b6ab-4d1d-85f2-7ee1c6e01247 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:13.621643 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cca77f0-b6ab-4d1d-85f2-7ee1c6e01247 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:13.630953 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-6cca77f0-b6ab-4d1d-85f2-7ee1c6e01247 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 31 12:21:13.630953 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:13.630953 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 31 12:21:13.630953 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 31 12:21:13.630953 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 31 12:21:13.630953 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 31 12:21:13.630953 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 31 12:21:13.630953 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 31 12:21:13.630953 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 31 12:21:13.630953 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:21:13.630953 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 31 12:21:13.630953 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:21:13.630953 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:13.630953 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:21:13.630953 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:21:13.630953 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:21:13.631917 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:21:13.631917 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:21:13.631917 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:21:13.631917 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:21:13.631917 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:21:13.631917 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:21:13.631917 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:21:13.631917 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 31 12:21:13.631917 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 31 12:21:13.631917 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 31 12:21:13.631917 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 31 12:21:13.631917 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:21:13.631917 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:21:13.631917 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:21:13.631917 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:21:13.631917 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:21:13.631917 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:21:13.633000 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:21:13.633000 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 31 12:21:13.633000 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:21:13.633000 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:21:13.633000 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:21:13.633000 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:21:13.633000 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:21:13.633000 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:21:13.633000 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:21:13.633000 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:21:13.633000 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 31 12:21:13.633000 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 31 12:21:13.633000 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 31 12:21:13.633000 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:21:13.633000 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 342/669] 10.222.0.15 () {56 vars in 1132 bytes} [Mon Oct 31 12:21:13 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:21:13.640524 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65c2174-c2fd-431d-8399-3896090b98be None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:13.694755 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:13.748661 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65c2174-c2fd-431d-8399-3896090b98be None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:13.784009 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48d381b-8388-4ef2-aefa-af0982fd385f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:13.784855 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48d381b-8388-4ef2-aefa-af0982fd385f None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:13.785275 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48d381b-8388-4ef2-aefa-af0982fd385f None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:13.785703 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48d381b-8388-4ef2-aefa-af0982fd385f None admin] PATH_INFO: `/v3/services` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:13.786479 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48d381b-8388-4ef2-aefa-af0982fd385f None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:13.786917 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48d381b-8388-4ef2-aefa-af0982fd385f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:13.788441 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48d381b-8388-4ef2-aefa-af0982fd385f None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:13.798596 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 328/670] 10.222.0.15 () {56 vars in 1126 bytes} [Mon Oct 31 12:21:13 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:21:13.800168 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cca77f0-b6ab-4d1d-85f2-7ee1c6e01247 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:13.906953 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cca77f0-b6ab-4d1d-85f2-7ee1c6e01247 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:13.942203 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c534f56-a1f4-43ae-8f13-a1268564c1c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:13.943122 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c534f56-a1f4-43ae-8f13-a1268564c1c0 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:13.943421 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c534f56-a1f4-43ae-8f13-a1268564c1c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:13.943687 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c534f56-a1f4-43ae-8f13-a1268564c1c0 None admin] PATH_INFO: `/v3/services` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:13.944312 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c534f56-a1f4-43ae-8f13-a1268564c1c0 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:13.944610 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c534f56-a1f4-43ae-8f13-a1268564c1c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:13.945950 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c534f56-a1f4-43ae-8f13-a1268564c1c0 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:13.958247 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 343/671] 10.222.0.15 () {56 vars in 1126 bytes} [Mon Oct 31 12:21:13 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:21:13.959702 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48d381b-8388-4ef2-aefa-af0982fd385f None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:14.063651 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48d381b-8388-4ef2-aefa-af0982fd385f None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:14.078111 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:14.098927 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316bb7a7-1b4a-4439-aa70-6f4bae7a2c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:14.099549 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316bb7a7-1b4a-4439-aa70-6f4bae7a2c4a None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:14.099824 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316bb7a7-1b4a-4439-aa70-6f4bae7a2c4a None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:14.100091 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316bb7a7-1b4a-4439-aa70-6f4bae7a2c4a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:14.100780 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316bb7a7-1b4a-4439-aa70-6f4bae7a2c4a None admin] RBAC: Adding query filter params (interface=public, service_id=0fc8aa558fe947e7bd8d2d7abb9708b1, region_id=RegionOne) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:14.101079 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316bb7a7-1b4a-4439-aa70-6f4bae7a2c4a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:14.102420 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316bb7a7-1b4a-4439-aa70-6f4bae7a2c4a None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:14.111269 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 329/672] 10.222.0.15 () {56 vars in 1251 bytes} [Mon Oct 31 12:21:13 2022] GET /identity/v3/endpoints?service_id=0fc8aa558fe947e7bd8d2d7abb9708b1&interface=public®ion_id=RegionOne => generated 194 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:21:14.113481 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c534f56-a1f4-43ae-8f13-a1268564c1c0 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:14.173238 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:14.223693 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c534f56-a1f4-43ae-8f13-a1268564c1c0 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:14.266768 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2389212f-580c-4277-8431-059f2712cde0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:14.267780 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2389212f-580c-4277-8431-059f2712cde0 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:14.268292 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2389212f-580c-4277-8431-059f2712cde0 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:14.269028 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2389212f-580c-4277-8431-059f2712cde0 None admin] PATH_INFO: `/v3/services` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:14.269831 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2389212f-580c-4277-8431-059f2712cde0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:14.271403 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2389212f-580c-4277-8431-059f2712cde0 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:14.279361 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 344/673] 10.222.0.15 () {56 vars in 1087 bytes} [Mon Oct 31 12:21:14 2022] GET /identity/v3/services => generated 1497 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:21:14.339379 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fb33f2-a064-4eb2-9e27-b0f6f79fb66f None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:14.339722 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fb33f2-a064-4eb2-9e27-b0f6f79fb66f None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:14.339933 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fb33f2-a064-4eb2-9e27-b0f6f79fb66f None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:14.341706 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 330/674] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:21:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:21:14.347563 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276e092b-27a6-463b-9757-c086100b1b34 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:14.347852 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276e092b-27a6-463b-9757-c086100b1b34 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:14.348114 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276e092b-27a6-463b-9757-c086100b1b34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:14.370974 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-276e092b-27a6-463b-9757-c086100b1b34 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:21:14.372269 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-276e092b-27a6-463b-9757-c086100b1b34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:21:14.447584 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 345/675] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:21:14 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 31 12:21:14.720841 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fb33f2-a064-4eb2-9e27-b0f6f79fb66f None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:14.831316 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fb33f2-a064-4eb2-9e27-b0f6f79fb66f None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:14.867570 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f11248-f2b6-44a5-8b52-bc5d1fb9d6e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:14.868195 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f11248-f2b6-44a5-8b52-bc5d1fb9d6e6 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:14.868468 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f11248-f2b6-44a5-8b52-bc5d1fb9d6e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:14.869474 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f11248-f2b6-44a5-8b52-bc5d1fb9d6e6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:14.869897 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f11248-f2b6-44a5-8b52-bc5d1fb9d6e6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:14.871268 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f11248-f2b6-44a5-8b52-bc5d1fb9d6e6 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:14.880354 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-c7f11248-f2b6-44a5-8b52-bc5d1fb9d6e6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 31 12:21:14.880354 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:14.880354 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 31 12:21:14.880354 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 31 12:21:14.880354 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 31 12:21:14.880354 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 31 12:21:14.880354 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 31 12:21:14.880354 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 31 12:21:14.880354 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 31 12:21:14.880354 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:21:14.880354 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 31 12:21:14.880354 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:21:14.880354 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:14.880354 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:21:14.880354 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:21:14.880354 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:21:14.880354 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:21:14.881364 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:21:14.881364 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:21:14.881364 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:21:14.881364 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:21:14.881364 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:21:14.881364 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:21:14.881364 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 31 12:21:14.881364 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 31 12:21:14.881364 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 31 12:21:14.881364 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 31 12:21:14.881364 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:21:14.881364 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:21:14.881364 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:21:14.881364 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:21:14.881364 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:21:14.881364 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:21:14.882239 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:21:14.882239 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:21:14.882239 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:21:14.882239 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:21:14.882239 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:21:14.882239 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:21:14.882239 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:21:14.882239 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:21:14.882239 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:21:14.882239 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:21:14.882239 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 31 12:21:14.882239 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 31 12:21:14.882239 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 31 12:21:14.882239 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:21:14.886556 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 331/676] 10.222.0.15 () {56 vars in 1126 bytes} [Mon Oct 31 12:21:14 2022] GET /identity/v3/services/object-store => generated 93 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:21:14.887937 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276e092b-27a6-463b-9757-c086100b1b34 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:14.929913 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27cbc25-7964-4629-9686-b9704905ac83 None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:14.930480 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27cbc25-7964-4629-9686-b9704905ac83 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:14.930959 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27cbc25-7964-4629-9686-b9704905ac83 None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:14.933336 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 332/677] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:21:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 31 12:21:14.938569 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d91d5e3-da8d-4e31-9979-be059aa80ad2 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:14.939087 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d91d5e3-da8d-4e31-9979-be059aa80ad2 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:14.939602 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d91d5e3-da8d-4e31-9979-be059aa80ad2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:14.941581 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:14.958262 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-1d91d5e3-da8d-4e31-9979-be059aa80ad2 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:21:14.959804 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-1d91d5e3-da8d-4e31-9979-be059aa80ad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:21:14.999578 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:15.008707 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276e092b-27a6-463b-9757-c086100b1b34 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:15.043376 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 333/678] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:21:14 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 31 12:21:15.061417 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a1ed0a-f763-4909-b1e7-3b70bfe78329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:15.062331 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a1ed0a-f763-4909-b1e7-3b70bfe78329 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:15.062813 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a1ed0a-f763-4909-b1e7-3b70bfe78329 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:15.063366 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a1ed0a-f763-4909-b1e7-3b70bfe78329 None admin] PATH_INFO: `/v3/services` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:15.064287 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a1ed0a-f763-4909-b1e7-3b70bfe78329 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:15.064802 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a1ed0a-f763-4909-b1e7-3b70bfe78329 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:15.066443 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a1ed0a-f763-4909-b1e7-3b70bfe78329 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:15.072025 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 346/679] 10.222.0.15 () {56 vars in 1122 bytes} [Mon Oct 31 12:21:14 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:21:15.077656 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d91d5e3-da8d-4e31-9979-be059aa80ad2 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:15.189551 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d91d5e3-da8d-4e31-9979-be059aa80ad2 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:15.230372 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9dce34-3468-4de3-8b0b-6f1e60cc3e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:15.230988 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9dce34-3468-4de3-8b0b-6f1e60cc3e4c None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:15.231302 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9dce34-3468-4de3-8b0b-6f1e60cc3e4c None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:15.231577 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9dce34-3468-4de3-8b0b-6f1e60cc3e4c None admin] PATH_INFO: `/v3/services` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:15.232202 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9dce34-3468-4de3-8b0b-6f1e60cc3e4c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:15.232506 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9dce34-3468-4de3-8b0b-6f1e60cc3e4c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:15.233960 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9dce34-3468-4de3-8b0b-6f1e60cc3e4c None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:15.239204 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 334/680] 10.222.0.15 () {56 vars in 1122 bytes} [Mon Oct 31 12:21:15 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 31 12:21:15.245535 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a1ed0a-f763-4909-b1e7-3b70bfe78329 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:15.317608 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9dce34-3468-4de3-8b0b-6f1e60cc3e4c None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:15.356846 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:15.364435 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a1ed0a-f763-4909-b1e7-3b70bfe78329 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:15.383442 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:15.411799 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba12c1f8-a077-479e-a455-09d5aef60efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:15.412960 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba12c1f8-a077-479e-a455-09d5aef60efc None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:15.413313 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba12c1f8-a077-479e-a455-09d5aef60efc None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:15.413658 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba12c1f8-a077-479e-a455-09d5aef60efc None admin] PATH_INFO: `/v3/endpoints` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:15.414434 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba12c1f8-a077-479e-a455-09d5aef60efc None admin] RBAC: Adding query filter params (interface=public, service_id=4f842180bbad485399fef6af786f0247, region_id=RegionOne) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:15.414813 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba12c1f8-a077-479e-a455-09d5aef60efc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:15.417563 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba12c1f8-a077-479e-a455-09d5aef60efc None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:15.425326 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 347/681] 10.222.0.15 () {56 vars in 1251 bytes} [Mon Oct 31 12:21:15 2022] GET /identity/v3/endpoints?service_id=4f842180bbad485399fef6af786f0247&interface=public®ion_id=RegionOne => generated 194 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:21:15.431110 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba12c1f8-a077-479e-a455-09d5aef60efc None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:15.437649 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9dce34-3468-4de3-8b0b-6f1e60cc3e4c None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:15.477228 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a045f38-1231-4220-bd19-b30e9d899907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:15.477929 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a045f38-1231-4220-bd19-b30e9d899907 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:15.478207 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a045f38-1231-4220-bd19-b30e9d899907 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:15.478990 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a045f38-1231-4220-bd19-b30e9d899907 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:15.489599 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a045f38-1231-4220-bd19-b30e9d899907 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 31 12:21:15.494998 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-3a045f38-1231-4220-bd19-b30e9d899907 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 31 12:21:15.494998 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:15.494998 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:21:15.494998 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:21:15.494998 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:21:15.494998 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:21:15.494998 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:21:15.494998 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:21:15.494998 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:21:15.494998 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:21:15.494998 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:21:15.494998 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:21:15.494998 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 31 12:21:15.494998 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 31 12:21:15.494998 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 31 12:21:15.494998 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 31 12:21:15.495955 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 31 12:21:15.495955 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 31 12:21:15.495955 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 31 12:21:15.495955 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 31 12:21:15.495955 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:21:15.495955 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:21:15.495955 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:21:15.495955 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:21:15.495955 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:21:15.495955 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:21:15.495955 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:21:15.495955 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:21:15.495955 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:21:15.495955 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:21:15.495955 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:21:15.495955 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:21:15.496867 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:21:15.496867 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:21:15.496867 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:21:15.496867 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:21:15.496867 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 31 12:21:15.496867 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 31 12:21:15.496867 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 31 12:21:15.496867 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 31 12:21:15.496867 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 31 12:21:15.496867 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 31 12:21:15.496867 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 31 12:21:15.496867 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:21:15.501125 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 335/682] 10.222.0.15 () {56 vars in 1120 bytes} [Mon Oct 31 12:21:15 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:21:15.502127 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a045f38-1231-4220-bd19-b30e9d899907 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:15.586381 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba12c1f8-a077-479e-a455-09d5aef60efc None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:15.627516 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:15.629372 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cfabff-9274-48b8-abec-698473b82fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:15.630034 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cfabff-9274-48b8-abec-698473b82fd6 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:15.630268 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cfabff-9274-48b8-abec-698473b82fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:15.630554 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cfabff-9274-48b8-abec-698473b82fd6 None admin] PATH_INFO: `/v3/services` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:15.631142 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cfabff-9274-48b8-abec-698473b82fd6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:15.632522 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cfabff-9274-48b8-abec-698473b82fd6 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:15.647187 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a045f38-1231-4220-bd19-b30e9d899907 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:15.647396 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 348/683] 10.222.0.15 () {56 vars in 1087 bytes} [Mon Oct 31 12:21:15 2022] GET /identity/v3/services => generated 1497 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 31 12:21:15.686988 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed511ec0-ed70-4e4d-a416-dd2c7f1e492e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:15.687726 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed511ec0-ed70-4e4d-a416-dd2c7f1e492e None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:15.688008 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed511ec0-ed70-4e4d-a416-dd2c7f1e492e None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:15.688276 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed511ec0-ed70-4e4d-a416-dd2c7f1e492e None admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:15.688948 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed511ec0-ed70-4e4d-a416-dd2c7f1e492e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:15.689249 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed511ec0-ed70-4e4d-a416-dd2c7f1e492e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:15.690637 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed511ec0-ed70-4e4d-a416-dd2c7f1e492e None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:15.701451 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 336/684] 10.222.0.15 () {56 vars in 1115 bytes} [Mon Oct 31 12:21:15 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 31 12:21:15.707456 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cfabff-9274-48b8-abec-698473b82fd6 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:15.766002 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:15.833415 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cfabff-9274-48b8-abec-698473b82fd6 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:15.847929 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd19f3a-ff41-4151-9ab2-7dd51101d0b2 None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:15.848548 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd19f3a-ff41-4151-9ab2-7dd51101d0b2 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:15.849058 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd19f3a-ff41-4151-9ab2-7dd51101d0b2 None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:15.850810 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 337/685] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:21:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:21:15.856603 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ae8226-82eb-4b9a-adc5-e23a3cbbf5b1 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:15.857104 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ae8226-82eb-4b9a-adc5-e23a3cbbf5b1 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:15.857588 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ae8226-82eb-4b9a-adc5-e23a3cbbf5b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:15.876667 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a80b800-947b-40d0-81e0-9cb647296b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:15.876667 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a80b800-947b-40d0-81e0-9cb647296b6b None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:15.876963 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a80b800-947b-40d0-81e0-9cb647296b6b None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:15.877174 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a80b800-947b-40d0-81e0-9cb647296b6b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:15.880440 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-77ae8226-82eb-4b9a-adc5-e23a3cbbf5b1 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:21:15.881848 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-77ae8226-82eb-4b9a-adc5-e23a3cbbf5b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:21:15.890605 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a80b800-947b-40d0-81e0-9cb647296b6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:15.892520 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a80b800-947b-40d0-81e0-9cb647296b6b None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:15.909226 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-3a80b800-947b-40d0-81e0-9cb647296b6b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:21:15.909226 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:15.909226 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:21:15.909226 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:21:15.909226 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:21:15.909226 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:21:15.909226 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:21:15.909226 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:21:15.909226 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:21:15.909226 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:21:15.909226 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:21:15.909226 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:21:15.909226 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 31 12:21:15.909226 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 31 12:21:15.909226 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 31 12:21:15.909226 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 31 12:21:15.910218 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:21:15.910218 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:21:15.910218 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:21:15.910218 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:21:15.910218 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:21:15.910218 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:21:15.910218 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:21:15.910218 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 31 12:21:15.910218 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:21:15.910218 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:21:15.910218 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:21:15.910218 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:21:15.910218 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:21:15.910218 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:21:15.910218 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:21:15.910218 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:21:15.910218 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 31 12:21:15.911595 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 31 12:21:15.911595 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:21:15.911595 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:21:15.911595 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 349/686] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Oct 31 12:21:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:21:15.917929 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a80b800-947b-40d0-81e0-9cb647296b6b None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:15.971280 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 338/687] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:21:15 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 31 12:21:16.035534 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:16.039701 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a80b800-947b-40d0-81e0-9cb647296b6b None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:16.079450 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4de76e-4082-4f9c-9ba3-ee9e66d263a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:16.080340 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4de76e-4082-4f9c-9ba3-ee9e66d263a1 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:16.080770 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4de76e-4082-4f9c-9ba3-ee9e66d263a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:16.081198 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4de76e-4082-4f9c-9ba3-ee9e66d263a1 None admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:16.082017 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4de76e-4082-4f9c-9ba3-ee9e66d263a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:16.082484 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4de76e-4082-4f9c-9ba3-ee9e66d263a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:16.084073 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4de76e-4082-4f9c-9ba3-ee9e66d263a1 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:16.097001 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 350/688] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Oct 31 12:21:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 31 12:21:16.104574 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ae8226-82eb-4b9a-adc5-e23a3cbbf5b1 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:16.203923 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:16.218069 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ae8226-82eb-4b9a-adc5-e23a3cbbf5b1 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:16.247831 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4de76e-4082-4f9c-9ba3-ee9e66d263a1 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:16.256872 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825a29a5-fee8-4892-b4e6-a4c4e4986fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:16.257501 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a29a5-fee8-4892-b4e6-a4c4e4986fe3 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:16.257777 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a29a5-fee8-4892-b4e6-a4c4e4986fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:16.258046 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a29a5-fee8-4892-b4e6-a4c4e4986fe3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:16.266503 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825a29a5-fee8-4892-b4e6-a4c4e4986fe3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:16.267956 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825a29a5-fee8-4892-b4e6-a4c4e4986fe3 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:16.281595 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-825a29a5-fee8-4892-b4e6-a4c4e4986fe3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 31 12:21:16.281595 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:16.281595 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:21:16.281595 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:21:16.281595 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:21:16.281595 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:21:16.281595 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:21:16.281595 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:21:16.281595 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:21:16.281595 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:21:16.281595 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:21:16.281595 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:21:16.281595 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 31 12:21:16.281595 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 31 12:21:16.281595 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 31 12:21:16.281595 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 31 12:21:16.282578 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:21:16.282578 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:21:16.282578 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 31 12:21:16.282578 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 31 12:21:16.282578 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 31 12:21:16.282578 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 31 12:21:16.282578 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:21:16.282578 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:21:16.282578 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:21:16.282578 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:21:16.282578 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:21:16.282578 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:21:16.282578 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:21:16.282578 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:21:16.282578 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:21:16.282578 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:21:16.282578 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:21:16.282578 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:21:16.283550 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:21:16.283550 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:21:16.283550 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 31 12:21:16.283550 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 31 12:21:16.283550 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 31 12:21:16.283550 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 31 12:21:16.283550 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 31 12:21:16.283550 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 31 12:21:16.283550 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 31 12:21:16.283550 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:21:16.283550 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 339/689] 10.222.0.15 () {56 vars in 1117 bytes} [Mon Oct 31 12:21:16 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:21:16.291955 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825a29a5-fee8-4892-b4e6-a4c4e4986fe3 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:16.366717 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4de76e-4082-4f9c-9ba3-ee9e66d263a1 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:16.406700 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373c05f3-d656-4d4d-9bb1-e11c5dd26c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:16.407360 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373c05f3-d656-4d4d-9bb1-e11c5dd26c30 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:16.407693 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373c05f3-d656-4d4d-9bb1-e11c5dd26c30 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:16.407967 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373c05f3-d656-4d4d-9bb1-e11c5dd26c30 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:16.408546 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373c05f3-d656-4d4d-9bb1-e11c5dd26c30 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:16.409882 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373c05f3-d656-4d4d-9bb1-e11c5dd26c30 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:16.419204 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-373c05f3-d656-4d4d-9bb1-e11c5dd26c30 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 31 12:21:16.419204 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:16.419204 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 31 12:21:16.419204 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 31 12:21:16.419204 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 31 12:21:16.419204 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 31 12:21:16.419204 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 31 12:21:16.419204 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 31 12:21:16.419204 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 31 12:21:16.419204 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:21:16.419204 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 31 12:21:16.419204 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:21:16.419204 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:16.419204 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:21:16.419204 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:21:16.419204 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:21:16.419204 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:21:16.420273 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:21:16.420273 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:21:16.420273 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:21:16.420273 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:21:16.420273 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:21:16.420273 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:21:16.420273 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 31 12:21:16.420273 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 31 12:21:16.420273 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 31 12:21:16.420273 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 31 12:21:16.420273 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:21:16.420273 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:21:16.420273 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:21:16.420273 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:21:16.420273 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:21:16.420273 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:21:16.421109 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:21:16.421109 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 31 12:21:16.421109 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:21:16.421109 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:21:16.421109 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:21:16.421109 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:21:16.421109 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:21:16.421109 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:21:16.421109 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:21:16.421109 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:21:16.421109 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 31 12:21:16.421109 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 31 12:21:16.421109 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 31 12:21:16.421109 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:21:16.425034 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 351/690] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Oct 31 12:21:16 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 31 12:21:16.426310 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373c05f3-d656-4d4d-9bb1-e11c5dd26c30 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:16.428360 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825a29a5-fee8-4892-b4e6-a4c4e4986fe3 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:16.451100 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:16.475832 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0154bc6c-674b-4559-9701-e15dfb1a53c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:16.476436 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0154bc6c-674b-4559-9701-e15dfb1a53c2 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:16.476709 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0154bc6c-674b-4559-9701-e15dfb1a53c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:16.476982 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0154bc6c-674b-4559-9701-e15dfb1a53c2 None admin] PATH_INFO: `/v3/users` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:16.478357 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0154bc6c-674b-4559-9701-e15dfb1a53c2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:16.478718 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0154bc6c-674b-4559-9701-e15dfb1a53c2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:16.480415 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0154bc6c-674b-4559-9701-e15dfb1a53c2 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:16.507812 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 340/691] 10.222.0.15 () {56 vars in 1149 bytes} [Mon Oct 31 12:21:16 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:21:16.507812 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290f3597-ea91-4769-9e0c-251bff020f56 None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:16.507812 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290f3597-ea91-4769-9e0c-251bff020f56 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:16.507812 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290f3597-ea91-4769-9e0c-251bff020f56 None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:16.507812 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 341/692] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:21:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:21:16.509773 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290f3597-ea91-4769-9e0c-251bff020f56 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:16.519814 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:16.551427 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373c05f3-d656-4d4d-9bb1-e11c5dd26c30 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:16.594885 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ba42a4-31f5-425c-be32-c4ad034cc106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:16.595537 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ba42a4-31f5-425c-be32-c4ad034cc106 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:16.595814 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ba42a4-31f5-425c-be32-c4ad034cc106 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:16.596082 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ba42a4-31f5-425c-be32-c4ad034cc106 None admin] PATH_INFO: `/v3/services` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:16.596700 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ba42a4-31f5-425c-be32-c4ad034cc106 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:16.597002 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ba42a4-31f5-425c-be32-c4ad034cc106 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:16.598376 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ba42a4-31f5-425c-be32-c4ad034cc106 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:16.608387 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 352/693] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Oct 31 12:21:16 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:21:16.608387 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28b5cf5-a0a3-4e8b-9dc9-01c03a8d58ac None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:16.608387 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28b5cf5-a0a3-4e8b-9dc9-01c03a8d58ac None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:16.608387 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28b5cf5-a0a3-4e8b-9dc9-01c03a8d58ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:16.624147 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-a28b5cf5-a0a3-4e8b-9dc9-01c03a8d58ac None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:21:16.625268 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-a28b5cf5-a0a3-4e8b-9dc9-01c03a8d58ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:21:16.633101 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290f3597-ea91-4769-9e0c-251bff020f56 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:16.672681 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434f17d3-600c-4490-85c0-fa23daf9f864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:16.673164 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434f17d3-600c-4490-85c0-fa23daf9f864 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:16.673445 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434f17d3-600c-4490-85c0-fa23daf9f864 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:16.673715 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434f17d3-600c-4490-85c0-fa23daf9f864 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:16.707539 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 353/694] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:21:16 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 31 12:21:16.707539 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28b5cf5-a0a3-4e8b-9dc9-01c03a8d58ac None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:16.721188 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:16.758907 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434f17d3-600c-4490-85c0-fa23daf9f864 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:16.761323 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434f17d3-600c-4490-85c0-fa23daf9f864 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:16.769714 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:16.818671 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28b5cf5-a0a3-4e8b-9dc9-01c03a8d58ac None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:16.857050 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d706922-02f6-4341-abb2-cdd9d8d54e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:16.857904 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d706922-02f6-4341-abb2-cdd9d8d54e69 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:16.858326 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d706922-02f6-4341-abb2-cdd9d8d54e69 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:16.858738 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d706922-02f6-4341-abb2-cdd9d8d54e69 None admin] PATH_INFO: `/v3/services` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:16.859523 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d706922-02f6-4341-abb2-cdd9d8d54e69 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:16.859982 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d706922-02f6-4341-abb2-cdd9d8d54e69 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:16.862227 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d706922-02f6-4341-abb2-cdd9d8d54e69 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:16.867705 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 354/695] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Oct 31 12:21:16 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:21:16.873743 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d706922-02f6-4341-abb2-cdd9d8d54e69 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:16.875964 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 342/696] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Oct 31 12:21:16 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 31 12:21:16.881476 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434f17d3-600c-4490-85c0-fa23daf9f864 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:16.959783 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:16.987612 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d706922-02f6-4341-abb2-cdd9d8d54e69 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:16.992544 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434f17d3-600c-4490-85c0-fa23daf9f864 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:17.001268 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:17.029000 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ec7296-c90e-45ab-8df8-7b09e73c4db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:17.029615 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ec7296-c90e-45ab-8df8-7b09e73c4db7 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:17.029892 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ec7296-c90e-45ab-8df8-7b09e73c4db7 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:17.030159 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ec7296-c90e-45ab-8df8-7b09e73c4db7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:17.030862 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ec7296-c90e-45ab-8df8-7b09e73c4db7 None admin] RBAC: Adding query filter params (interface=public, service_id=6b55f272892a47258f4f550cbe1d0d08, region_id=RegionOne) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:17.031189 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ec7296-c90e-45ab-8df8-7b09e73c4db7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:17.032593 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ec7296-c90e-45ab-8df8-7b09e73c4db7 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:17.034961 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201c6360-a1f9-4dbe-9155-a83611158131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:17.035625 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201c6360-a1f9-4dbe-9155-a83611158131 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:17.035905 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201c6360-a1f9-4dbe-9155-a83611158131 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:17.036174 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201c6360-a1f9-4dbe-9155-a83611158131 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:17.042807 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 355/697] 10.222.0.15 () {56 vars in 1251 bytes} [Mon Oct 31 12:21:16 2022] GET /identity/v3/endpoints?service_id=6b55f272892a47258f4f550cbe1d0d08&interface=public®ion_id=RegionOne => generated 194 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 31 12:21:17.043585 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ec7296-c90e-45ab-8df8-7b09e73c4db7 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:17.047262 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201c6360-a1f9-4dbe-9155-a83611158131 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:17.048734 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201c6360-a1f9-4dbe-9155-a83611158131 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:17.064587 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-201c6360-a1f9-4dbe-9155-a83611158131 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:21:17.064587 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:17.064587 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:21:17.064587 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:21:17.064587 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:21:17.064587 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:21:17.064587 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:21:17.064587 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:21:17.064587 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:21:17.064587 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:21:17.064587 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:21:17.064587 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:21:17.064587 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 31 12:21:17.064587 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 31 12:21:17.064587 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 31 12:21:17.064587 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 31 12:21:17.065528 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:21:17.065528 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:21:17.065528 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:21:17.065528 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:21:17.065528 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:21:17.065528 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:21:17.065528 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:21:17.065528 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:21:17.065528 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:21:17.065528 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:21:17.065528 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:21:17.065528 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:21:17.065528 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:21:17.065528 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:21:17.065528 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:21:17.065528 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:21:17.065528 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 31 12:21:17.066469 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 31 12:21:17.066469 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:21:17.066469 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:21:17.066469 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 343/698] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Oct 31 12:21:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 31 12:21:17.066469 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201c6360-a1f9-4dbe-9155-a83611158131 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:17.153630 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ec7296-c90e-45ab-8df8-7b09e73c4db7 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:17.174689 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201c6360-a1f9-4dbe-9155-a83611158131 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:17.190578 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5742efb0-489a-491a-8dac-4383d35c9188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:17.191229 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5742efb0-489a-491a-8dac-4383d35c9188 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:17.191510 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5742efb0-489a-491a-8dac-4383d35c9188 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:17.191784 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5742efb0-489a-491a-8dac-4383d35c9188 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:17.192361 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5742efb0-489a-491a-8dac-4383d35c9188 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:17.193711 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5742efb0-489a-491a-8dac-4383d35c9188 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:17.203387 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-5742efb0-489a-491a-8dac-4383d35c9188 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 31 12:21:17.203387 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:17.203387 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 31 12:21:17.203387 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 31 12:21:17.203387 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 31 12:21:17.203387 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 31 12:21:17.203387 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 31 12:21:17.203387 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 31 12:21:17.203387 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 31 12:21:17.203387 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:21:17.203387 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 31 12:21:17.203387 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:21:17.203387 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:17.203387 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:21:17.203387 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:21:17.203387 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:21:17.204325 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:21:17.204325 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:21:17.204325 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:21:17.204325 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:21:17.204325 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:21:17.204325 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:21:17.204325 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:21:17.204325 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 31 12:21:17.204325 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 31 12:21:17.204325 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 31 12:21:17.204325 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 31 12:21:17.204325 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:21:17.204325 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:21:17.204325 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:21:17.204325 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:21:17.204325 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:21:17.204325 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:21:17.205239 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:21:17.205239 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 31 12:21:17.205239 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:21:17.205239 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:21:17.205239 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:21:17.205239 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:21:17.205239 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:21:17.205239 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:21:17.205239 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:21:17.205239 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:21:17.205239 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 31 12:21:17.205239 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 31 12:21:17.205239 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 31 12:21:17.205239 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:21:17.205239 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 356/699] 10.222.0.15 () {56 vars in 1132 bytes} [Mon Oct 31 12:21:17 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:21:17.206038 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5742efb0-489a-491a-8dac-4383d35c9188 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:17.210801 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:17.215132 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26b24d7-a670-40b9-879a-cc8222d8a955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:17.216031 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26b24d7-a670-40b9-879a-cc8222d8a955 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:17.216558 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26b24d7-a670-40b9-879a-cc8222d8a955 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:17.217088 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26b24d7-a670-40b9-879a-cc8222d8a955 None admin] PATH_INFO: `/v3/services` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:17.217916 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26b24d7-a670-40b9-879a-cc8222d8a955 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:17.219615 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26b24d7-a670-40b9-879a-cc8222d8a955 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:17.225996 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 344/700] 10.222.0.15 () {56 vars in 1087 bytes} [Mon Oct 31 12:21:17 2022] GET /identity/v3/services => generated 1497 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:21:17.226977 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26b24d7-a670-40b9-879a-cc8222d8a955 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:17.254233 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:17.347408 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5742efb0-489a-491a-8dac-4383d35c9188 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:17.374590 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26b24d7-a670-40b9-879a-cc8222d8a955 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:17.393295 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b61e8fd-c277-49a9-9b04-d3e5f34a0635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:17.394785 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b61e8fd-c277-49a9-9b04-d3e5f34a0635 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:17.395554 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b61e8fd-c277-49a9-9b04-d3e5f34a0635 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:17.399111 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b61e8fd-c277-49a9-9b04-d3e5f34a0635 None admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:17.400430 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b61e8fd-c277-49a9-9b04-d3e5f34a0635 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:17.401089 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b61e8fd-c277-49a9-9b04-d3e5f34a0635 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:17.403835 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b61e8fd-c277-49a9-9b04-d3e5f34a0635 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:17.418049 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a20cafb-a441-4ceb-9b50-f64355fe5ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:17.418666 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a20cafb-a441-4ceb-9b50-f64355fe5ad9 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:17.418941 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a20cafb-a441-4ceb-9b50-f64355fe5ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:17.419241 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a20cafb-a441-4ceb-9b50-f64355fe5ad9 None admin] PATH_INFO: `/v3/services` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:17.419873 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a20cafb-a441-4ceb-9b50-f64355fe5ad9 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:17.420173 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a20cafb-a441-4ceb-9b50-f64355fe5ad9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:17.421577 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a20cafb-a441-4ceb-9b50-f64355fe5ad9 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:17.421962 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 357/701] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Oct 31 12:21:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:21:17.428645 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 345/702] 10.222.0.15 () {56 vars in 1126 bytes} [Mon Oct 31 12:21:17 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:21:17.428824 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b61e8fd-c277-49a9-9b04-d3e5f34a0635 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:17.433468 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a20cafb-a441-4ceb-9b50-f64355fe5ad9 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:17.501193 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:17.545426 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b61e8fd-c277-49a9-9b04-d3e5f34a0635 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:17.549090 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:17.569482 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a20cafb-a441-4ceb-9b50-f64355fe5ad9 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:17.592439 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c34c7dc-42a1-496d-8b0a-b9e2744d02c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:17.593348 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c34c7dc-42a1-496d-8b0a-b9e2744d02c0 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:17.593629 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c34c7dc-42a1-496d-8b0a-b9e2744d02c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:17.593896 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c34c7dc-42a1-496d-8b0a-b9e2744d02c0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:17.604110 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c34c7dc-42a1-496d-8b0a-b9e2744d02c0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:17.606442 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c34c7dc-42a1-496d-8b0a-b9e2744d02c0 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:17.612996 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499e81f9-d66a-499c-95a9-4f0cebb0c197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:17.613712 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499e81f9-d66a-499c-95a9-4f0cebb0c197 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:17.614084 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499e81f9-d66a-499c-95a9-4f0cebb0c197 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:17.614446 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499e81f9-d66a-499c-95a9-4f0cebb0c197 None admin] PATH_INFO: `/v3/services` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:17.615254 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499e81f9-d66a-499c-95a9-4f0cebb0c197 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:17.615651 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499e81f9-d66a-499c-95a9-4f0cebb0c197 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:17.617099 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499e81f9-d66a-499c-95a9-4f0cebb0c197 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:17.620940 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-1c34c7dc-42a1-496d-8b0a-b9e2744d02c0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 31 12:21:17.620940 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:17.620940 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:21:17.620940 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:21:17.620940 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:21:17.620940 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:21:17.620940 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:21:17.620940 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:21:17.620940 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:21:17.620940 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:21:17.620940 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:21:17.620940 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:21:17.620940 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 31 12:21:17.620940 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 31 12:21:17.620940 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 31 12:21:17.622222 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 31 12:21:17.622222 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:21:17.622222 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:21:17.622222 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:21:17.622222 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:21:17.622222 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:21:17.622222 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:21:17.622222 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:21:17.622222 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 31 12:21:17.622222 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:21:17.622222 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:21:17.622222 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:21:17.622222 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:21:17.622222 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:21:17.622222 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:21:17.622222 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:21:17.622222 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:21:17.624138 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 31 12:21:17.624138 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 31 12:21:17.624138 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 31 12:21:17.624138 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 31 12:21:17.624138 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 31 12:21:17.624138 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 31 12:21:17.624138 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 31 12:21:17.624138 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:21:17.624971 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 346/704] 10.222.0.15 () {56 vars in 1126 bytes} [Mon Oct 31 12:21:17 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 31 12:21:17.628002 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499e81f9-d66a-499c-95a9-4f0cebb0c197 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:17.631185 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 358/704] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Oct 31 12:21:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 31 12:21:17.633790 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c34c7dc-42a1-496d-8b0a-b9e2744d02c0 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:17.737625 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499e81f9-d66a-499c-95a9-4f0cebb0c197 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:17.773487 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a7c38f-3f24-40ec-a935-c0603caec3bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:17.773814 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c34c7dc-42a1-496d-8b0a-b9e2744d02c0 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:17.774252 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a7c38f-3f24-40ec-a935-c0603caec3bf None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:17.774644 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a7c38f-3f24-40ec-a935-c0603caec3bf None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:17.775510 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a7c38f-3f24-40ec-a935-c0603caec3bf None admin] PATH_INFO: `/v3/projects` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:17.776259 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a7c38f-3f24-40ec-a935-c0603caec3bf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:17.776638 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a7c38f-3f24-40ec-a935-c0603caec3bf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:17.778118 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a7c38f-3f24-40ec-a935-c0603caec3bf None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:17.791721 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 347/705] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Oct 31 12:21:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:21:17.797216 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a7c38f-3f24-40ec-a935-c0603caec3bf None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:17.808702 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:17.810273 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:17.825072 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece44a97-2ee0-4e9f-b4df-ab3f33f80e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:17.826114 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece44a97-2ee0-4e9f-b4df-ab3f33f80e0e None admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:17.826674 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece44a97-2ee0-4e9f-b4df-ab3f33f80e0e None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:17.826951 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece44a97-2ee0-4e9f-b4df-ab3f33f80e0e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:17.827882 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece44a97-2ee0-4e9f-b4df-ab3f33f80e0e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:17.829917 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece44a97-2ee0-4e9f-b4df-ab3f33f80e0e None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:17.859378 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 359/706] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:21:17 2022] POST /identity/v3/endpoints => generated 347 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 31 12:21:17.920533 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a7c38f-3f24-40ec-a935-c0603caec3bf None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:17.972899 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9549c4-c9d9-4e7c-8ded-0ca0325c0606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:17.973588 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9549c4-c9d9-4e7c-8ded-0ca0325c0606 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:17.973872 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9549c4-c9d9-4e7c-8ded-0ca0325c0606 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:17.974143 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9549c4-c9d9-4e7c-8ded-0ca0325c0606 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:17.974896 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9549c4-c9d9-4e7c-8ded-0ca0325c0606 None admin] RBAC: Adding query filter params (role.id=92746653d6b94cf5a4e22d9efd5d4e4f, scope.project.id=2c32444cef584bce966b38203ce128b2, user.id=bf4faacf7df248d28f32d7014dadcc07) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:17.975223 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9549c4-c9d9-4e7c-8ded-0ca0325c0606 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:17.976699 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9549c4-c9d9-4e7c-8ded-0ca0325c0606 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:17.984636 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 348/707] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Oct 31 12:21:17 2022] GET /identity/v3/role_assignments?user.id=bf4faacf7df248d28f32d7014dadcc07&scope.project.id=2c32444cef584bce966b38203ce128b2&role.id=92746653d6b94cf5a4e22d9efd5d4e4f => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:21:18.011713 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4815e4b-e9f5-4172-ab68-c1a1212b90d9 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:18.012346 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4815e4b-e9f5-4172-ab68-c1a1212b90d9 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:18.012346 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4815e4b-e9f5-4172-ab68-c1a1212b90d9 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:18.013943 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 360/708] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:21:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:21:18.031212 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2f2827-7fd3-4440-9361-64cbacf86871 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:18.031212 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2f2827-7fd3-4440-9361-64cbacf86871 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:18.031212 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2f2827-7fd3-4440-9361-64cbacf86871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:18.048080 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-3e2f2827-7fd3-4440-9361-64cbacf86871 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:21:18.049831 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-3e2f2827-7fd3-4440-9361-64cbacf86871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:21:18.126220 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:18.132093 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 349/709] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:21:18 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 31 12:21:18.403697 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4815e4b-e9f5-4172-ab68-c1a1212b90d9 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:18.512231 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4815e4b-e9f5-4172-ab68-c1a1212b90d9 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:18.547289 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97c2433-8a2e-49fa-8044-3f194ed283fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:18.547923 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97c2433-8a2e-49fa-8044-3f194ed283fb None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:18.548201 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97c2433-8a2e-49fa-8044-3f194ed283fb None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:18.548472 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97c2433-8a2e-49fa-8044-3f194ed283fb None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:18.549055 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97c2433-8a2e-49fa-8044-3f194ed283fb None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:18.550406 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97c2433-8a2e-49fa-8044-3f194ed283fb None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:18.560410 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-d97c2433-8a2e-49fa-8044-3f194ed283fb None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 31 12:21:18.560410 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:18.560410 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 31 12:21:18.560410 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 31 12:21:18.560410 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 31 12:21:18.560410 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 31 12:21:18.560410 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 31 12:21:18.560410 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 31 12:21:18.560410 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 31 12:21:18.560410 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:21:18.560410 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 31 12:21:18.560410 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:21:18.560410 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:18.560410 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:21:18.560410 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:21:18.560410 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:21:18.560410 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:21:18.561417 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:21:18.561417 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:21:18.561417 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:21:18.561417 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:21:18.561417 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:21:18.561417 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:21:18.561417 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 31 12:21:18.561417 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 31 12:21:18.561417 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 31 12:21:18.561417 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 31 12:21:18.561417 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:21:18.561417 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:21:18.561417 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:21:18.561417 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:21:18.561417 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:21:18.561417 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:21:18.562445 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:21:18.562445 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 31 12:21:18.562445 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:21:18.562445 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:21:18.562445 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:21:18.562445 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:21:18.562445 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:21:18.562445 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:21:18.562445 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:21:18.562445 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:21:18.562445 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 31 12:21:18.562445 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 31 12:21:18.562445 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 31 12:21:18.562445 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:21:18.562445 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 361/710] 10.222.0.15 () {56 vars in 1126 bytes} [Mon Oct 31 12:21:18 2022] GET /identity/v3/services/object-store => generated 93 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:21:18.570367 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2f2827-7fd3-4440-9361-64cbacf86871 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:18.676679 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2f2827-7fd3-4440-9361-64cbacf86871 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:18.714913 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45918b5f-8062-46c1-bca2-720e8cb5493b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:18.715549 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45918b5f-8062-46c1-bca2-720e8cb5493b None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:18.715826 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45918b5f-8062-46c1-bca2-720e8cb5493b None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:18.716099 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45918b5f-8062-46c1-bca2-720e8cb5493b None admin] PATH_INFO: `/v3/services` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:18.716722 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45918b5f-8062-46c1-bca2-720e8cb5493b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:18.717020 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45918b5f-8062-46c1-bca2-720e8cb5493b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:18.718380 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45918b5f-8062-46c1-bca2-720e8cb5493b None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:18.723644 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 350/711] 10.222.0.15 () {56 vars in 1122 bytes} [Mon Oct 31 12:21:18 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:21:18.732845 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97c2433-8a2e-49fa-8044-3f194ed283fb None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:18.781679 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:18.843379 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97c2433-8a2e-49fa-8044-3f194ed283fb None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:18.879225 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cec001-ebb1-4f3c-a08e-5514e3f3bdf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:18.880346 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cec001-ebb1-4f3c-a08e-5514e3f3bdf6 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:18.880623 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cec001-ebb1-4f3c-a08e-5514e3f3bdf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:18.880894 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cec001-ebb1-4f3c-a08e-5514e3f3bdf6 None admin] PATH_INFO: `/v3/services` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:18.881548 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cec001-ebb1-4f3c-a08e-5514e3f3bdf6 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:18.881851 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cec001-ebb1-4f3c-a08e-5514e3f3bdf6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:18.883295 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cec001-ebb1-4f3c-a08e-5514e3f3bdf6 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:18.890063 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 362/712] 10.222.0.15 () {56 vars in 1122 bytes} [Mon Oct 31 12:21:18 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:21:18.895892 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45918b5f-8062-46c1-bca2-720e8cb5493b None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:18.969596 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:19.010764 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45918b5f-8062-46c1-bca2-720e8cb5493b None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:19.057905 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e50a5bf-4037-4375-a4d7-a5f2ddd009a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:19.058511 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e50a5bf-4037-4375-a4d7-a5f2ddd009a2 None admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:19.058775 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e50a5bf-4037-4375-a4d7-a5f2ddd009a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:19.059046 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e50a5bf-4037-4375-a4d7-a5f2ddd009a2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:19.059904 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e50a5bf-4037-4375-a4d7-a5f2ddd009a2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:19.061316 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e50a5bf-4037-4375-a4d7-a5f2ddd009a2 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:19.087803 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 351/713] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:21:18 2022] POST /identity/v3/endpoints => generated 349 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:21:19.503090 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b9d463-f820-4d2c-8714-c1534d5d9c09 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:19.503642 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b9d463-f820-4d2c-8714-c1534d5d9c09 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:19.503642 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b9d463-f820-4d2c-8714-c1534d5d9c09 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:19.505240 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 363/714] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:21:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:21:19.510604 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7a2ff6-1ac5-45de-b79e-6f2e0c07704a None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:19.510894 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7a2ff6-1ac5-45de-b79e-6f2e0c07704a None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:19.511194 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7a2ff6-1ac5-45de-b79e-6f2e0c07704a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:19.530419 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-1e7a2ff6-1ac5-45de-b79e-6f2e0c07704a None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:21:19.531642 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-1e7a2ff6-1ac5-45de-b79e-6f2e0c07704a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:21:19.609928 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 352/715] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:21:19 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 31 12:21:19.875473 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b9d463-f820-4d2c-8714-c1534d5d9c09 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:19.983664 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b9d463-f820-4d2c-8714-c1534d5d9c09 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:19.993525 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:20.033067 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6226c7-bacf-4e9f-8194-af4ed319bc0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:20.033786 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6226c7-bacf-4e9f-8194-af4ed319bc0b None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:20.034069 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6226c7-bacf-4e9f-8194-af4ed319bc0b None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:20.034340 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6226c7-bacf-4e9f-8194-af4ed319bc0b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:20.034926 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6226c7-bacf-4e9f-8194-af4ed319bc0b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:20.036336 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6226c7-bacf-4e9f-8194-af4ed319bc0b None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:20.045920 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-8f6226c7-bacf-4e9f-8194-af4ed319bc0b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 31 12:21:20.045920 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:20.045920 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 31 12:21:20.045920 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 31 12:21:20.045920 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 31 12:21:20.045920 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 31 12:21:20.045920 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 31 12:21:20.045920 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 31 12:21:20.045920 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 31 12:21:20.045920 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:21:20.045920 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 31 12:21:20.045920 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:21:20.045920 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:20.045920 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:21:20.045920 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:21:20.045920 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:21:20.045920 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:21:20.046923 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:21:20.046923 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:21:20.046923 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:21:20.046923 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:21:20.046923 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:21:20.046923 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:21:20.046923 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 31 12:21:20.046923 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 31 12:21:20.046923 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 31 12:21:20.046923 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 31 12:21:20.046923 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:21:20.046923 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:21:20.046923 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:21:20.046923 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:21:20.046923 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:21:20.046923 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:21:20.047807 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:21:20.047807 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 31 12:21:20.047807 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:21:20.047807 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:21:20.047807 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:21:20.047807 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:21:20.047807 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:21:20.047807 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:21:20.047807 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:21:20.047807 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:21:20.047807 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 31 12:21:20.047807 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 31 12:21:20.047807 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 31 12:21:20.047807 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:21:20.047807 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 364/716] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Oct 31 12:21:19 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:21:20.053071 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7a2ff6-1ac5-45de-b79e-6f2e0c07704a None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:20.105228 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:20.170063 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7a2ff6-1ac5-45de-b79e-6f2e0c07704a None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:20.205332 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b0b0cb-48ba-493f-acb9-4f91e85baf0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:20.206327 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b0b0cb-48ba-493f-acb9-4f91e85baf0e None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:20.206801 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b0b0cb-48ba-493f-acb9-4f91e85baf0e None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:20.207347 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b0b0cb-48ba-493f-acb9-4f91e85baf0e None admin] PATH_INFO: `/v3/services` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:20.208159 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b0b0cb-48ba-493f-acb9-4f91e85baf0e None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:20.208667 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b0b0cb-48ba-493f-acb9-4f91e85baf0e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:20.210202 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b0b0cb-48ba-493f-acb9-4f91e85baf0e None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:20.215970 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 353/717] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Oct 31 12:21:20 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:21:20.221753 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6226c7-bacf-4e9f-8194-af4ed319bc0b None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:20.238587 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e1d32f-60ac-42c3-b16f-c9c3915bc90c None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:20.238909 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e1d32f-60ac-42c3-b16f-c9c3915bc90c None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:20.239136 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e1d32f-60ac-42c3-b16f-c9c3915bc90c None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:20.240759 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 354/718] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:21:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 31 12:21:20.247603 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681ac462-9abe-4680-ae1a-c232507b43d1 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:20.247886 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681ac462-9abe-4680-ae1a-c232507b43d1 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:20.248159 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681ac462-9abe-4680-ae1a-c232507b43d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:20.267946 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-681ac462-9abe-4680-ae1a-c232507b43d1 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:21:20.269386 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-681ac462-9abe-4680-ae1a-c232507b43d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:21:20.339109 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6226c7-bacf-4e9f-8194-af4ed319bc0b None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:20.339329 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 355/719] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:21:20 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 31 12:21:20.340657 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb170de2-eb46-4012-9d45-affda13c9dbc None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:20.341003 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb170de2-eb46-4012-9d45-affda13c9dbc None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:20.341280 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb170de2-eb46-4012-9d45-affda13c9dbc None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:20.342581 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 356/720] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:21:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:21:20.347824 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e0b0d2-0cac-4d63-9225-5badf43d2118 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:20.348099 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e0b0d2-0cac-4d63-9225-5badf43d2118 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:20.348365 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e0b0d2-0cac-4d63-9225-5badf43d2118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:20.368276 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-72e0b0d2-0cac-4d63-9225-5badf43d2118 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:21:20.370285 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-72e0b0d2-0cac-4d63-9225-5badf43d2118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:21:20.383401 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411c1720-8177-4292-9b71-a2133d85ab85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:20.385100 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411c1720-8177-4292-9b71-a2133d85ab85 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:20.385462 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411c1720-8177-4292-9b71-a2133d85ab85 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:20.385802 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411c1720-8177-4292-9b71-a2133d85ab85 None admin] PATH_INFO: `/v3/services` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:20.386720 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411c1720-8177-4292-9b71-a2133d85ab85 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:20.387021 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411c1720-8177-4292-9b71-a2133d85ab85 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:20.388642 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411c1720-8177-4292-9b71-a2133d85ab85 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:20.399069 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 365/721] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Oct 31 12:21:20 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:21:20.400975 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411c1720-8177-4292-9b71-a2133d85ab85 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:20.414639 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:20.440436 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:20.459206 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 357/722] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:21:20 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 31 12:21:20.515449 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411c1720-8177-4292-9b71-a2133d85ab85 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:20.550007 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cebe30-5458-4be6-9887-c46b0eb6c6df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:20.550602 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cebe30-5458-4be6-9887-c46b0eb6c6df None admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:20.550874 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cebe30-5458-4be6-9887-c46b0eb6c6df None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:20.551143 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cebe30-5458-4be6-9887-c46b0eb6c6df None admin] PATH_INFO: `/v3/endpoints` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:20.551814 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cebe30-5458-4be6-9887-c46b0eb6c6df None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:20.553142 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cebe30-5458-4be6-9887-c46b0eb6c6df None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:20.578118 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 366/723] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:21:20 2022] POST /identity/v3/endpoints => generated 346 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 31 12:21:20.620806 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e0b0d2-0cac-4d63-9225-5badf43d2118 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:20.734419 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e0b0d2-0cac-4d63-9225-5badf43d2118 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:20.742370 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cebe30-5458-4be6-9887-c46b0eb6c6df None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:20.781638 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed9309c-e914-4965-bc5b-a27cd2e35989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:20.782279 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed9309c-e914-4965-bc5b-a27cd2e35989 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:20.782557 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed9309c-e914-4965-bc5b-a27cd2e35989 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:20.782839 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed9309c-e914-4965-bc5b-a27cd2e35989 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:20.783584 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed9309c-e914-4965-bc5b-a27cd2e35989 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:20.785104 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed9309c-e914-4965-bc5b-a27cd2e35989 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:20.794964 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-2ed9309c-e914-4965-bc5b-a27cd2e35989 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 31 12:21:20.794964 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:20.794964 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 31 12:21:20.794964 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 31 12:21:20.794964 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 31 12:21:20.794964 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 31 12:21:20.794964 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 31 12:21:20.794964 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 31 12:21:20.794964 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 31 12:21:20.794964 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:21:20.794964 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 31 12:21:20.794964 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:21:20.794964 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:20.794964 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:21:20.794964 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:21:20.794964 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:21:20.794964 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:21:20.796016 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:21:20.796016 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:21:20.796016 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:21:20.796016 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:21:20.796016 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:21:20.796016 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:21:20.796016 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 31 12:21:20.796016 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 31 12:21:20.796016 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 31 12:21:20.796016 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 31 12:21:20.796016 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:21:20.796016 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:21:20.796016 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:21:20.796016 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:21:20.796016 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:21:20.796016 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:21:20.796840 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:21:20.796840 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:21:20.796840 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:21:20.796840 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:21:20.796840 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:21:20.796840 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:21:20.796840 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:21:20.796840 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:21:20.796840 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:21:20.796840 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:21:20.796840 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 31 12:21:20.796840 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 31 12:21:20.796840 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 31 12:21:20.796840 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:21:20.796840 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 358/724] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Oct 31 12:21:20 2022] GET /identity/v3/services/compute => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:21:20.802094 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed9309c-e914-4965-bc5b-a27cd2e35989 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:20.843608 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:20.855729 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:20.870318 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cebe30-5458-4be6-9887-c46b0eb6c6df None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:20.918678 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c7dfac-f0c1-429e-b308-357a276ac4d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:20.920578 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c7dfac-f0c1-429e-b308-357a276ac4d6 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:20.920862 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c7dfac-f0c1-429e-b308-357a276ac4d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:20.921136 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c7dfac-f0c1-429e-b308-357a276ac4d6 None admin] PATH_INFO: `/v3/services/image` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:20.921723 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c7dfac-f0c1-429e-b308-357a276ac4d6 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:20.923091 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c7dfac-f0c1-429e-b308-357a276ac4d6 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:20.928435 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed9309c-e914-4965-bc5b-a27cd2e35989 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:20.938451 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-22c7dfac-f0c1-429e-b308-357a276ac4d6 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 31 12:21:20.938451 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:20.938451 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 31 12:21:20.938451 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 31 12:21:20.938451 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 31 12:21:20.938451 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 31 12:21:20.938451 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 31 12:21:20.938451 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 31 12:21:20.938451 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 31 12:21:20.938451 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:21:20.938451 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 31 12:21:20.938451 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:21:20.938451 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:20.938451 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:21:20.938451 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:21:20.938451 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:21:20.938451 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:21:20.939499 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:21:20.939499 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:21:20.939499 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:21:20.939499 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:21:20.939499 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:21:20.939499 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:21:20.939499 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 31 12:21:20.939499 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 31 12:21:20.939499 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 31 12:21:20.939499 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 31 12:21:20.939499 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:21:20.939499 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:21:20.939499 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:21:20.939499 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:21:20.939499 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:21:20.939499 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:21:20.940326 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:21:20.940326 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 31 12:21:20.940326 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:21:20.940326 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:21:20.940326 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:21:20.940326 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:21:20.940326 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:21:20.940326 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:21:20.940326 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:21:20.940326 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:21:20.940326 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 31 12:21:20.940326 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 31 12:21:20.940326 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 31 12:21:20.940326 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:21:20.941162 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 367/725] 10.222.0.15 () {56 vars in 1105 bytes} [Mon Oct 31 12:21:20 2022] GET /identity/v3/services/image => generated 86 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 31 12:21:20.947727 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c7dfac-f0c1-429e-b308-357a276ac4d6 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:20.973876 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef4c12a-847e-44aa-b0bd-4fa97fb25619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:20.974760 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef4c12a-847e-44aa-b0bd-4fa97fb25619 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:20.975321 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef4c12a-847e-44aa-b0bd-4fa97fb25619 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:20.975725 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef4c12a-847e-44aa-b0bd-4fa97fb25619 None admin] PATH_INFO: `/v3/services` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:20.976568 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef4c12a-847e-44aa-b0bd-4fa97fb25619 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:20.977004 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef4c12a-847e-44aa-b0bd-4fa97fb25619 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:20.978660 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef4c12a-847e-44aa-b0bd-4fa97fb25619 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:20.984291 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 359/726] 10.222.0.15 () {56 vars in 1112 bytes} [Mon Oct 31 12:21:20 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 31 12:21:20.991086 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef4c12a-847e-44aa-b0bd-4fa97fb25619 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:21.082306 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c7dfac-f0c1-429e-b308-357a276ac4d6 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:21.122220 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1339767e-c0d3-48bd-9557-6ba6c3e0fd10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:21.122829 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1339767e-c0d3-48bd-9557-6ba6c3e0fd10 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:21.123104 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1339767e-c0d3-48bd-9557-6ba6c3e0fd10 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:21.123399 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1339767e-c0d3-48bd-9557-6ba6c3e0fd10 None admin] PATH_INFO: `/v3/services` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:21.124018 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1339767e-c0d3-48bd-9557-6ba6c3e0fd10 None admin] RBAC: Adding query filter params (name=image) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:21.124421 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1339767e-c0d3-48bd-9557-6ba6c3e0fd10 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:21.125780 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1339767e-c0d3-48bd-9557-6ba6c3e0fd10 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:21.129865 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:21.135237 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 368/727] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Oct 31 12:21:20 2022] GET /identity/v3/services?name=image => generated 122 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:21:21.138036 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1339767e-c0d3-48bd-9557-6ba6c3e0fd10 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:21.146439 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef4c12a-847e-44aa-b0bd-4fa97fb25619 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:21.163894 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:21.186449 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845d950a-353e-459f-be53-9099957da5f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:21.187243 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845d950a-353e-459f-be53-9099957da5f3 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:21.187677 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845d950a-353e-459f-be53-9099957da5f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:21.188102 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845d950a-353e-459f-be53-9099957da5f3 None admin] PATH_INFO: `/v3/services` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:21.189263 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845d950a-353e-459f-be53-9099957da5f3 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:21.189263 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845d950a-353e-459f-be53-9099957da5f3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:21.190793 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845d950a-353e-459f-be53-9099957da5f3 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:21.203191 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 360/728] 10.222.0.15 () {56 vars in 1112 bytes} [Mon Oct 31 12:21:20 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:21:21.269315 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1339767e-c0d3-48bd-9557-6ba6c3e0fd10 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:21.309105 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9155f9a-65af-4772-addd-af70b1a9ac01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:21.309787 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9155f9a-65af-4772-addd-af70b1a9ac01 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:21.310069 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9155f9a-65af-4772-addd-af70b1a9ac01 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:21.310341 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9155f9a-65af-4772-addd-af70b1a9ac01 None admin] PATH_INFO: `/v3/services` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:21.310981 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9155f9a-65af-4772-addd-af70b1a9ac01 None admin] RBAC: Adding query filter params (type=image) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:21.311316 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9155f9a-65af-4772-addd-af70b1a9ac01 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:21.312721 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9155f9a-65af-4772-addd-af70b1a9ac01 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:21.318460 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 369/729] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Oct 31 12:21:21 2022] GET /identity/v3/services?type=image => generated 122 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:21:21.346754 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a864d3ad-2ea3-44ef-9a83-4f2c7ccac91f None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:21.347042 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a864d3ad-2ea3-44ef-9a83-4f2c7ccac91f None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:21.347346 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a864d3ad-2ea3-44ef-9a83-4f2c7ccac91f None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:21.348993 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 361/730] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:21:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:21:21.355442 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb589e54-9956-418d-90ab-2ffdd14e97ae None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:21.355722 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb589e54-9956-418d-90ab-2ffdd14e97ae None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:21.355988 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb589e54-9956-418d-90ab-2ffdd14e97ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:21.375875 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-cb589e54-9956-418d-90ab-2ffdd14e97ae None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:21:21.377212 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-cb589e54-9956-418d-90ab-2ffdd14e97ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:21:21.461401 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 370/731] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:21:21 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 31 12:21:21.726668 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a864d3ad-2ea3-44ef-9a83-4f2c7ccac91f None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:21.840494 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a864d3ad-2ea3-44ef-9a83-4f2c7ccac91f None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:21.874894 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e28bc6-a401-4731-8198-5c143fd2c8c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:21.875548 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e28bc6-a401-4731-8198-5c143fd2c8c9 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:21.875822 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e28bc6-a401-4731-8198-5c143fd2c8c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:21.876090 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e28bc6-a401-4731-8198-5c143fd2c8c9 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:21.876664 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e28bc6-a401-4731-8198-5c143fd2c8c9 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:21.878012 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e28bc6-a401-4731-8198-5c143fd2c8c9 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:21.887468 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-68e28bc6-a401-4731-8198-5c143fd2c8c9 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 31 12:21:21.887468 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:21.887468 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 31 12:21:21.887468 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 31 12:21:21.887468 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 31 12:21:21.887468 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 31 12:21:21.887468 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 31 12:21:21.887468 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 31 12:21:21.887468 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 31 12:21:21.887468 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:21:21.887468 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 31 12:21:21.887468 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:21:21.887468 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:21.887468 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:21:21.887468 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:21:21.887468 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:21:21.887468 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:21:21.888479 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:21:21.888479 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:21:21.888479 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:21:21.888479 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:21:21.888479 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:21:21.888479 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:21:21.888479 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 31 12:21:21.888479 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 31 12:21:21.888479 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 31 12:21:21.888479 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 31 12:21:21.888479 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:21:21.888479 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:21:21.888479 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:21:21.888479 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:21:21.888479 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:21:21.888479 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:21:21.889319 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:21:21.889319 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:21:21.889319 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:21:21.889319 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:21:21.889319 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:21:21.889319 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:21:21.889319 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:21:21.889319 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:21:21.889319 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:21:21.889319 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:21:21.889319 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 31 12:21:21.889319 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 31 12:21:21.889319 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 31 12:21:21.889319 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:21:21.889319 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 362/732] 10.222.0.15 () {56 vars in 1126 bytes} [Mon Oct 31 12:21:21 2022] GET /identity/v3/services/object-store => generated 93 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:21:21.894968 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb589e54-9956-418d-90ab-2ffdd14e97ae None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:21.911003 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:22.001352 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb589e54-9956-418d-90ab-2ffdd14e97ae None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:22.038142 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e083ac47-62d3-487e-85f7-93f593fe7981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:22.038759 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e083ac47-62d3-487e-85f7-93f593fe7981 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:22.039029 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e083ac47-62d3-487e-85f7-93f593fe7981 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:22.039321 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e083ac47-62d3-487e-85f7-93f593fe7981 None admin] PATH_INFO: `/v3/services` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:22.039941 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e083ac47-62d3-487e-85f7-93f593fe7981 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:22.040240 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e083ac47-62d3-487e-85f7-93f593fe7981 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:22.042271 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e083ac47-62d3-487e-85f7-93f593fe7981 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:22.047640 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 371/733] 10.222.0.15 () {56 vars in 1122 bytes} [Mon Oct 31 12:21:21 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:21:22.053283 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e28bc6-a401-4731-8198-5c143fd2c8c9 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:22.079237 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:22.158505 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e28bc6-a401-4731-8198-5c143fd2c8c9 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:22.196124 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2b3caa-51e7-4686-a4cf-89376bc0b56a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:22.197299 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2b3caa-51e7-4686-a4cf-89376bc0b56a None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:22.197587 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2b3caa-51e7-4686-a4cf-89376bc0b56a None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:22.197838 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2b3caa-51e7-4686-a4cf-89376bc0b56a None admin] PATH_INFO: `/v3/services` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:22.198461 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2b3caa-51e7-4686-a4cf-89376bc0b56a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:22.198761 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2b3caa-51e7-4686-a4cf-89376bc0b56a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:22.200179 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2b3caa-51e7-4686-a4cf-89376bc0b56a None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:22.205724 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 363/734] 10.222.0.15 () {56 vars in 1122 bytes} [Mon Oct 31 12:21:22 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:21:22.211397 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e083ac47-62d3-487e-85f7-93f593fe7981 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:22.295847 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:22.317936 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e083ac47-62d3-487e-85f7-93f593fe7981 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:22.352723 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081f0f64-20fa-4f97-a459-71ef544dd827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:22.353582 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081f0f64-20fa-4f97-a459-71ef544dd827 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:22.353854 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081f0f64-20fa-4f97-a459-71ef544dd827 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:22.354118 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081f0f64-20fa-4f97-a459-71ef544dd827 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:22.354812 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081f0f64-20fa-4f97-a459-71ef544dd827 None admin] RBAC: Adding query filter params (interface=admin, service_id=4f842180bbad485399fef6af786f0247, region_id=RegionOne) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:22.355110 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081f0f64-20fa-4f97-a459-71ef544dd827 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:22.356478 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081f0f64-20fa-4f97-a459-71ef544dd827 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:22.361725 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 372/735] 10.222.0.15 () {56 vars in 1249 bytes} [Mon Oct 31 12:21:22 2022] GET /identity/v3/endpoints?service_id=4f842180bbad485399fef6af786f0247&interface=admin®ion_id=RegionOne => generated 193 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:21:22.367352 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2b3caa-51e7-4686-a4cf-89376bc0b56a None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:22.455431 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:22.476345 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2b3caa-51e7-4686-a4cf-89376bc0b56a None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:22.513119 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a6f120-4756-4182-9fc2-5bbaa00ccdb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:22.514092 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a6f120-4756-4182-9fc2-5bbaa00ccdb9 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:22.514574 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a6f120-4756-4182-9fc2-5bbaa00ccdb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:22.515044 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a6f120-4756-4182-9fc2-5bbaa00ccdb9 None admin] PATH_INFO: `/v3/services` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:22.515861 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a6f120-4756-4182-9fc2-5bbaa00ccdb9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:22.517412 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a6f120-4756-4182-9fc2-5bbaa00ccdb9 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:22.524477 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 364/736] 10.222.0.15 () {56 vars in 1087 bytes} [Mon Oct 31 12:21:22 2022] GET /identity/v3/services => generated 1497 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:21:22.919336 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f7a0cc-b9a2-45d0-8875-55409fdd3cf9 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:22.919852 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f7a0cc-b9a2-45d0-8875-55409fdd3cf9 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:22.919852 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f7a0cc-b9a2-45d0-8875-55409fdd3cf9 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:22.921526 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 373/737] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:21:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:21:22.926828 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13044448-f70f-44dc-9041-767f644ce693 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:22.927116 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13044448-f70f-44dc-9041-767f644ce693 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:22.927428 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13044448-f70f-44dc-9041-767f644ce693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:22.946016 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-13044448-f70f-44dc-9041-767f644ce693 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:21:22.947745 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-13044448-f70f-44dc-9041-767f644ce693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:21:23.017733 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 365/738] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:21:22 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 31 12:21:23.285943 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f7a0cc-b9a2-45d0-8875-55409fdd3cf9 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:23.389445 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f7a0cc-b9a2-45d0-8875-55409fdd3cf9 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:23.428873 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afbe706-1261-4ab7-8f88-170ccc6e1f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:23.429512 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afbe706-1261-4ab7-8f88-170ccc6e1f5a None admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:23.429785 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afbe706-1261-4ab7-8f88-170ccc6e1f5a None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:23.430051 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afbe706-1261-4ab7-8f88-170ccc6e1f5a None admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:23.430722 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afbe706-1261-4ab7-8f88-170ccc6e1f5a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:23.432305 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afbe706-1261-4ab7-8f88-170ccc6e1f5a None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:23.435111 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: WARNING keystone.server.flask.common [None req-0afbe706-1261-4ab7-8f88-170ccc6e1f5a None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 31 12:21:23.453030 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:23.456973 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.sql.core [None req-0afbe706-1261-4ab7-8f88-170ccc6e1f5a None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 31 12:21:23.456973 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 31 12:21:23.456973 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [parameters: {'id': 'c08ea5ba41b14cad982ddd718aee76bf', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 31 12:21:23.456973 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81491) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 31 12:21:23.466941 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-0afbe706-1261-4ab7-8f88-170ccc6e1f5a None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c08ea5ba41b14cad982ddd718aee76bf.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c08ea5ba41b14cad982ddd718aee76bf. Oct 31 12:21:23.466941 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:23.466941 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 31 12:21:23.466941 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 31 12:21:23.466941 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 31 12:21:23.466941 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 31 12:21:23.466941 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 31 12:21:23.466941 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application result = self._query(query) Oct 31 12:21:23.466941 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 31 12:21:23.466941 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application conn.query(q) Oct 31 12:21:23.466941 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 31 12:21:23.466941 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 31 12:21:23.466941 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 31 12:21:23.467880 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application result.read() Oct 31 12:21:23.467880 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 31 12:21:23.467880 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 31 12:21:23.467880 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 31 12:21:23.467880 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 31 12:21:23.467880 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 31 12:21:23.467880 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 31 12:21:23.467880 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 31 12:21:23.467880 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 31 12:21:23.467880 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 31 12:21:23.467880 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:21:23.467880 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 31 12:21:23.467880 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:21:23.467880 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:23.467880 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 31 12:21:23.467880 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 31 12:21:23.467880 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 31 12:21:23.467880 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 31 12:21:23.468923 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 31 12:21:23.468923 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application next(self.gen) Oct 31 12:21:23.468923 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Oct 31 12:21:23.468923 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application yield resource Oct 31 12:21:23.468923 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 31 12:21:23.468923 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application next(self.gen) Oct 31 12:21:23.468923 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Oct 31 12:21:23.468923 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self.session.rollback() Oct 31 12:21:23.468923 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 31 12:21:23.468923 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self.force_reraise() Oct 31 12:21:23.468923 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 31 12:21:23.468923 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise self.value Oct 31 12:21:23.468923 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Oct 31 12:21:23.468923 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 31 12:21:23.468923 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Oct 31 12:21:23.468923 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application session.commit() Oct 31 12:21:23.468923 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 31 12:21:23.469796 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 31 12:21:23.469796 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 31 12:21:23.469796 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self._prepare_impl() Oct 31 12:21:23.469796 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 31 12:21:23.469796 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self.session.flush() Oct 31 12:21:23.469796 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 31 12:21:23.469796 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self._flush(objects) Oct 31 12:21:23.469796 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 31 12:21:23.469796 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 31 12:21:23.469796 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 31 12:21:23.469796 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application compat.raise_( Oct 31 12:21:23.469796 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 31 12:21:23.469796 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception Oct 31 12:21:23.469796 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 31 12:21:23.469796 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application flush_context.execute() Oct 31 12:21:23.469796 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 31 12:21:23.469796 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rec.execute(self) Oct 31 12:21:23.470688 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 31 12:21:23.470688 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 31 12:21:23.470688 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 31 12:21:23.470688 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 31 12:21:23.470688 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 31 12:21:23.470688 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 31 12:21:23.470688 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 31 12:21:23.470688 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 31 12:21:23.470688 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 31 12:21:23.470688 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 31 12:21:23.470688 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 31 12:21:23.470688 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 31 12:21:23.470688 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 31 12:21:23.470688 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 31 12:21:23.470688 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 31 12:21:23.471521 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 31 12:21:23.471521 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 31 12:21:23.471521 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception Oct 31 12:21:23.471521 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 31 12:21:23.471521 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 31 12:21:23.471521 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 31 12:21:23.471521 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 31 12:21:23.471521 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 31 12:21:23.471521 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application result = self._query(query) Oct 31 12:21:23.471521 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 31 12:21:23.471521 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application conn.query(q) Oct 31 12:21:23.471521 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 31 12:21:23.471521 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 31 12:21:23.471521 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 31 12:21:23.471521 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application result.read() Oct 31 12:21:23.471521 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 31 12:21:23.471521 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 31 12:21:23.472393 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 31 12:21:23.472393 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 31 12:21:23.472393 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 31 12:21:23.472393 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 31 12:21:23.472393 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 31 12:21:23.472393 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 31 12:21:23.472393 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 31 12:21:23.472393 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 31 12:21:23.472393 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [parameters: {'id': 'c08ea5ba41b14cad982ddd718aee76bf', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 31 12:21:23.472393 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 31 12:21:23.472393 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:21:23.472393 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 31 12:21:23.472393 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:21:23.472393 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:23.472393 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 31 12:21:23.473427 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 31 12:21:23.473427 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 31 12:21:23.473427 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 31 12:21:23.473427 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Oct 31 12:21:23.473427 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:21:23.473427 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 31 12:21:23.473427 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:21:23.473427 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:23.473427 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:21:23.473427 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:21:23.473427 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:21:23.473427 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:21:23.473427 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:21:23.473427 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:21:23.473427 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:21:23.473427 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:21:23.473427 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:21:23.474529 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:21:23.474529 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Oct 31 12:21:23.474529 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Oct 31 12:21:23.474529 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:21:23.474529 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:21:23.474529 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 31 12:21:23.474529 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 31 12:21:23.474529 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c08ea5ba41b14cad982ddd718aee76bf. Oct 31 12:21:23.474529 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:21:23.476965 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 374/739] 10.222.0.15 () {60 vars in 1140 bytes} [Mon Oct 31 12:21:23 2022] POST /identity/v3/projects => generated 262 bytes in 191 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:21:23.483697 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13044448-f70f-44dc-9041-767f644ce693 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:23.527136 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bf436f-c1e8-44a3-9065-ab051fa7575d None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:23.527510 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bf436f-c1e8-44a3-9065-ab051fa7575d None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:23.527742 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bf436f-c1e8-44a3-9065-ab051fa7575d None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:23.532297 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 375/740] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:21:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:21:23.537264 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cc75bd-fd5b-40f2-8bee-ae1fae2e09b9 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:23.537484 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cc75bd-fd5b-40f2-8bee-ae1fae2e09b9 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:23.537750 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cc75bd-fd5b-40f2-8bee-ae1fae2e09b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:23.557394 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-22cc75bd-fd5b-40f2-8bee-ae1fae2e09b9 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:21:23.559004 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-22cc75bd-fd5b-40f2-8bee-ae1fae2e09b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:21:23.585147 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:23.609211 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13044448-f70f-44dc-9041-767f644ce693 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:23.633613 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 376/741] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:21:23 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 31 12:21:23.643054 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cc78e3-31c5-4399-90d8-2fe847e7667a None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:23.643054 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cc78e3-31c5-4399-90d8-2fe847e7667a None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:23.643054 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cc78e3-31c5-4399-90d8-2fe847e7667a None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:23.643054 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 377/742] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:21:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:21:23.644626 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdbaf3a-9cb8-43e2-9e69-b2a247fe811a None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:23.644901 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdbaf3a-9cb8-43e2-9e69-b2a247fe811a None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:23.645173 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdbaf3a-9cb8-43e2-9e69-b2a247fe811a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:23.651508 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762442af-1faa-48c3-942c-14a97731e696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:23.652175 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762442af-1faa-48c3-942c-14a97731e696 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:23.653382 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762442af-1faa-48c3-942c-14a97731e696 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:23.653656 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762442af-1faa-48c3-942c-14a97731e696 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:23.661705 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762442af-1faa-48c3-942c-14a97731e696 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:23.663269 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762442af-1faa-48c3-942c-14a97731e696 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:23.663482 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-7fdbaf3a-9cb8-43e2-9e69-b2a247fe811a None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:21:23.664659 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-7fdbaf3a-9cb8-43e2-9e69-b2a247fe811a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:21:23.679199 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-762442af-1faa-48c3-942c-14a97731e696 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 31 12:21:23.679199 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:23.679199 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:21:23.679199 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:21:23.679199 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:21:23.679199 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:21:23.679199 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:21:23.679199 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:21:23.679199 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:21:23.679199 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:21:23.679199 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:21:23.679199 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:21:23.679199 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 31 12:21:23.679199 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 31 12:21:23.679199 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 31 12:21:23.680085 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 31 12:21:23.680085 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:21:23.680085 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:21:23.680085 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:21:23.680085 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:21:23.680085 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:21:23.680085 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:21:23.680085 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:21:23.680085 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:21:23.680085 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:21:23.680085 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:21:23.680085 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:21:23.680085 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:21:23.680085 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:21:23.680085 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:21:23.680085 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:21:23.680085 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:21:23.681193 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 31 12:21:23.681193 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 31 12:21:23.681193 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 31 12:21:23.681193 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 31 12:21:23.681193 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 31 12:21:23.681193 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 31 12:21:23.681193 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 31 12:21:23.681193 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:21:23.683420 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 366/743] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Oct 31 12:21:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:21:23.686318 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762442af-1faa-48c3-942c-14a97731e696 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:23.757255 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 378/744] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:21:23 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 31 12:21:23.804637 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762442af-1faa-48c3-942c-14a97731e696 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:23.840166 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba677e4-d181-487e-a333-930070753b93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:23.840824 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba677e4-d181-487e-a333-930070753b93 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:23.841101 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba677e4-d181-487e-a333-930070753b93 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:23.841369 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba677e4-d181-487e-a333-930070753b93 None admin] PATH_INFO: `/v3/projects` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:23.842076 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba677e4-d181-487e-a333-930070753b93 None admin] RBAC: Adding query filter params (name=service) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:23.842380 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba677e4-d181-487e-a333-930070753b93 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:23.843822 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba677e4-d181-487e-a333-930070753b93 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:23.858841 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:23.863298 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 367/745] 10.222.0.15 () {56 vars in 1112 bytes} [Mon Oct 31 12:21:23 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 31 12:21:23.913642 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdbaf3a-9cb8-43e2-9e69-b2a247fe811a None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:23.977008 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:24.033551 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdbaf3a-9cb8-43e2-9e69-b2a247fe811a None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:24.059440 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba677e4-d181-487e-a333-930070753b93 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:24.070581 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723e5b97-3075-4adf-8ff4-6e34f1314719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:24.071205 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723e5b97-3075-4adf-8ff4-6e34f1314719 None admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:24.071487 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723e5b97-3075-4adf-8ff4-6e34f1314719 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:24.071864 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723e5b97-3075-4adf-8ff4-6e34f1314719 None admin] PATH_INFO: `/v3/services` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:24.072510 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723e5b97-3075-4adf-8ff4-6e34f1314719 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:24.073984 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723e5b97-3075-4adf-8ff4-6e34f1314719 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:24.092926 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 379/746] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Oct 31 12:21:23 2022] POST /identity/v3/services => generated 242 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:21:24.182314 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba677e4-d181-487e-a333-930070753b93 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:24.229620 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad2269f-7664-4960-bab2-782acae7b406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:24.230499 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad2269f-7664-4960-bab2-782acae7b406 None admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:24.230932 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad2269f-7664-4960-bab2-782acae7b406 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:24.231391 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad2269f-7664-4960-bab2-782acae7b406 None admin] PATH_INFO: `/v3/services` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:24.232296 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad2269f-7664-4960-bab2-782acae7b406 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:24.233970 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad2269f-7664-4960-bab2-782acae7b406 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:24.255544 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 368/747] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Oct 31 12:21:24 2022] POST /identity/v3/services => generated 242 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 31 12:21:24.763738 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b5be65-8ee0-405f-b1ca-17be2e904342 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:24.764447 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b5be65-8ee0-405f-b1ca-17be2e904342 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:24.764927 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b5be65-8ee0-405f-b1ca-17be2e904342 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:24.766922 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 380/748] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:21:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:21:24.772808 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af71c7e2-c745-4d96-bbf0-2c5849bcf8df None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:24.773669 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af71c7e2-c745-4d96-bbf0-2c5849bcf8df None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:24.774146 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af71c7e2-c745-4d96-bbf0-2c5849bcf8df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:24.793612 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-af71c7e2-c745-4d96-bbf0-2c5849bcf8df None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:21:24.795141 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-af71c7e2-c745-4d96-bbf0-2c5849bcf8df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:21:24.848573 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:24.879324 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 369/749] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:21:24 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 31 12:21:25.148310 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b5be65-8ee0-405f-b1ca-17be2e904342 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:25.263444 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b5be65-8ee0-405f-b1ca-17be2e904342 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:25.282099 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:25.300407 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53108d0a-6685-4ebd-b941-63e7689cc73e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:25.301357 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53108d0a-6685-4ebd-b941-63e7689cc73e None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:25.301829 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53108d0a-6685-4ebd-b941-63e7689cc73e None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:25.302290 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53108d0a-6685-4ebd-b941-63e7689cc73e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:25.303147 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53108d0a-6685-4ebd-b941-63e7689cc73e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:25.304781 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53108d0a-6685-4ebd-b941-63e7689cc73e None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:25.314559 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-53108d0a-6685-4ebd-b941-63e7689cc73e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 31 12:21:25.314559 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:25.314559 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 31 12:21:25.314559 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 31 12:21:25.314559 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 31 12:21:25.314559 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 31 12:21:25.314559 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 31 12:21:25.314559 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 31 12:21:25.314559 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 31 12:21:25.314559 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:21:25.314559 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 31 12:21:25.314559 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:21:25.314559 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:25.314559 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:21:25.314559 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:21:25.314559 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:21:25.314559 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:21:25.315561 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:21:25.315561 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:21:25.315561 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:21:25.315561 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:21:25.315561 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:21:25.315561 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:21:25.315561 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 31 12:21:25.315561 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 31 12:21:25.315561 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 31 12:21:25.315561 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 31 12:21:25.315561 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:21:25.315561 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:21:25.315561 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:21:25.315561 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:21:25.315561 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:21:25.315561 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:21:25.316381 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:21:25.316381 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 31 12:21:25.316381 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:21:25.316381 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:21:25.316381 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:21:25.316381 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:21:25.316381 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:21:25.316381 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:21:25.316381 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:21:25.316381 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:21:25.316381 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 31 12:21:25.316381 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 31 12:21:25.316381 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 31 12:21:25.316381 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:21:25.319031 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 381/750] 10.222.0.15 () {56 vars in 1126 bytes} [Mon Oct 31 12:21:25 2022] GET /identity/v3/services/object-store => generated 93 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:21:25.332574 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af71c7e2-c745-4d96-bbf0-2c5849bcf8df None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:25.437043 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af71c7e2-c745-4d96-bbf0-2c5849bcf8df None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:25.473216 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d781ab6-9f05-4c30-b649-d48c7a16a807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:25.473824 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d781ab6-9f05-4c30-b649-d48c7a16a807 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:25.474104 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d781ab6-9f05-4c30-b649-d48c7a16a807 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:25.474427 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d781ab6-9f05-4c30-b649-d48c7a16a807 None admin] PATH_INFO: `/v3/services` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:25.475052 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d781ab6-9f05-4c30-b649-d48c7a16a807 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:25.475380 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d781ab6-9f05-4c30-b649-d48c7a16a807 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:25.476803 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d781ab6-9f05-4c30-b649-d48c7a16a807 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:25.486165 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 370/751] 10.222.0.15 () {56 vars in 1122 bytes} [Mon Oct 31 12:21:25 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:21:25.487987 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53108d0a-6685-4ebd-b941-63e7689cc73e None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:25.593859 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53108d0a-6685-4ebd-b941-63e7689cc73e None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:25.628905 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00eaf81-c4c5-403f-9d64-a67565c735ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:25.629514 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00eaf81-c4c5-403f-9d64-a67565c735ac None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:25.629825 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00eaf81-c4c5-403f-9d64-a67565c735ac None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:25.630116 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00eaf81-c4c5-403f-9d64-a67565c735ac None admin] PATH_INFO: `/v3/services` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:25.630739 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00eaf81-c4c5-403f-9d64-a67565c735ac None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:25.631042 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00eaf81-c4c5-403f-9d64-a67565c735ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:25.632427 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00eaf81-c4c5-403f-9d64-a67565c735ac None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:25.638407 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 382/752] 10.222.0.15 () {56 vars in 1122 bytes} [Mon Oct 31 12:21:25 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:21:25.644543 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d781ab6-9f05-4c30-b649-d48c7a16a807 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:25.694865 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:25.750953 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d781ab6-9f05-4c30-b649-d48c7a16a807 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:25.786226 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f268f75b-9092-4ee0-b76f-d34f06e41828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:25.786875 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f268f75b-9092-4ee0-b76f-d34f06e41828 None admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:25.787099 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f268f75b-9092-4ee0-b76f-d34f06e41828 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:25.787389 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f268f75b-9092-4ee0-b76f-d34f06e41828 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:25.788050 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f268f75b-9092-4ee0-b76f-d34f06e41828 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:25.789397 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f268f75b-9092-4ee0-b76f-d34f06e41828 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:25.812620 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 371/753] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:21:25 2022] POST /identity/v3/endpoints => generated 325 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:21:26.206387 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef57761-353f-4405-8ad2-9d471e104ca6 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:26.206899 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef57761-353f-4405-8ad2-9d471e104ca6 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:26.206899 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef57761-353f-4405-8ad2-9d471e104ca6 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:26.208529 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 383/754] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:21:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:21:26.214653 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b200f384-2ee8-43ec-b1c9-565eb10ff44a None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:26.217842 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b200f384-2ee8-43ec-b1c9-565eb10ff44a None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:26.217842 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b200f384-2ee8-43ec-b1c9-565eb10ff44a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:26.234875 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-b200f384-2ee8-43ec-b1c9-565eb10ff44a None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:21:26.236116 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-b200f384-2ee8-43ec-b1c9-565eb10ff44a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:21:26.317326 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 372/755] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:21:26 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 31 12:21:26.425130 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45f2b75-b27c-4e17-bd4d-893ea53c1966 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:26.425751 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45f2b75-b27c-4e17-bd4d-893ea53c1966 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:26.426216 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45f2b75-b27c-4e17-bd4d-893ea53c1966 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:26.428039 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 384/756] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:21:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:21:26.433489 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f813a3-2698-44a6-9023-77840466f863 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:26.433999 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f813a3-2698-44a6-9023-77840466f863 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:26.434461 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f813a3-2698-44a6-9023-77840466f863 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:26.454927 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-a9f813a3-2698-44a6-9023-77840466f863 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:21:26.456460 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-a9f813a3-2698-44a6-9023-77840466f863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:21:26.486161 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:26.488088 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b0305a-3a86-49bb-a785-a00b66be093a None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:26.488603 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b0305a-3a86-49bb-a785-a00b66be093a None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:26.489044 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b0305a-3a86-49bb-a785-a00b66be093a None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:26.490918 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 385/757] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:21:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:21:26.496562 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54ac219-a6fa-40a0-bd14-e0ee2b716189 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:26.497132 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54ac219-a6fa-40a0-bd14-e0ee2b716189 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:26.497604 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54ac219-a6fa-40a0-bd14-e0ee2b716189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:26.516167 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-f54ac219-a6fa-40a0-bd14-e0ee2b716189 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:21:26.517667 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-f54ac219-a6fa-40a0-bd14-e0ee2b716189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:21:26.535579 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 373/758] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:21:26 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 31 12:21:26.567418 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:26.599190 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 386/759] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:21:26 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 31 12:21:26.599399 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f813a3-2698-44a6-9023-77840466f863 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:26.705861 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f813a3-2698-44a6-9023-77840466f863 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:26.741126 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451c0834-f065-4d34-bb4c-423afe817af5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:26.741753 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451c0834-f065-4d34-bb4c-423afe817af5 None admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:26.742027 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451c0834-f065-4d34-bb4c-423afe817af5 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:26.742303 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451c0834-f065-4d34-bb4c-423afe817af5 None admin] PATH_INFO: `/v3/users` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:26.742968 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451c0834-f065-4d34-bb4c-423afe817af5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:26.745021 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451c0834-f065-4d34-bb4c-423afe817af5 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:26.747690 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: WARNING keystone.server.flask.common [None req-451c0834-f065-4d34-bb4c-423afe817af5 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 31 12:21:26.761873 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.sql.core [None req-451c0834-f065-4d34-bb4c-423afe817af5 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 31 12:21:26.761873 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 31 12:21:26.761873 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [parameters: {'user_id': 'd924aea0f5b04436a86ee73ad0ed4c1d', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 31 12:21:26.761873 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81492) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 31 12:21:26.778854 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-451c0834-f065-4d34-bb4c-423afe817af5 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 31 12:21:26.778854 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:26.778854 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 31 12:21:26.778854 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 31 12:21:26.778854 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 31 12:21:26.778854 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 31 12:21:26.778854 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 31 12:21:26.778854 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application result = self._query(query) Oct 31 12:21:26.778854 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 31 12:21:26.778854 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application conn.query(q) Oct 31 12:21:26.778854 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 31 12:21:26.778854 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 31 12:21:26.778854 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 31 12:21:26.778854 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application result.read() Oct 31 12:21:26.778854 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 31 12:21:26.779787 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 31 12:21:26.779787 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 31 12:21:26.779787 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 31 12:21:26.779787 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 31 12:21:26.779787 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 31 12:21:26.779787 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 31 12:21:26.779787 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 31 12:21:26.779787 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 31 12:21:26.779787 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:21:26.779787 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 31 12:21:26.779787 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:21:26.779787 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:26.779787 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 31 12:21:26.779787 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 31 12:21:26.779787 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Oct 31 12:21:26.779787 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Oct 31 12:21:26.779787 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 31 12:21:26.779787 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application next(self.gen) Oct 31 12:21:26.780730 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Oct 31 12:21:26.780730 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application yield resource Oct 31 12:21:26.780730 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 31 12:21:26.780730 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application next(self.gen) Oct 31 12:21:26.780730 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Oct 31 12:21:26.780730 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application self.session.rollback() Oct 31 12:21:26.780730 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 31 12:21:26.780730 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application self.force_reraise() Oct 31 12:21:26.780730 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 31 12:21:26.780730 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise self.value Oct 31 12:21:26.780730 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Oct 31 12:21:26.780730 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 31 12:21:26.780730 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Oct 31 12:21:26.780730 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application session.commit() Oct 31 12:21:26.780730 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 31 12:21:26.780730 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 31 12:21:26.780730 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 31 12:21:26.781596 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application self._prepare_impl() Oct 31 12:21:26.781596 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 31 12:21:26.781596 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application self.session.flush() Oct 31 12:21:26.781596 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 31 12:21:26.781596 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application self._flush(objects) Oct 31 12:21:26.781596 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 31 12:21:26.781596 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 31 12:21:26.781596 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 31 12:21:26.781596 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application compat.raise_( Oct 31 12:21:26.781596 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 31 12:21:26.781596 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception Oct 31 12:21:26.781596 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 31 12:21:26.781596 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application flush_context.execute() Oct 31 12:21:26.781596 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 31 12:21:26.781596 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rec.execute(self) Oct 31 12:21:26.781596 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 31 12:21:26.781596 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 31 12:21:26.783708 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 31 12:21:26.783708 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 31 12:21:26.783708 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Oct 31 12:21:26.783708 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application result = connection._execute_20( Oct 31 12:21:26.783708 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 31 12:21:26.783708 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 31 12:21:26.783708 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 31 12:21:26.783708 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 31 12:21:26.783708 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 31 12:21:26.783708 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 31 12:21:26.783708 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 31 12:21:26.783708 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 31 12:21:26.783708 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 31 12:21:26.783708 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 31 12:21:26.783708 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 31 12:21:26.783708 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception Oct 31 12:21:26.785668 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 31 12:21:26.785668 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 31 12:21:26.785668 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 31 12:21:26.785668 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 31 12:21:26.785668 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 31 12:21:26.785668 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application result = self._query(query) Oct 31 12:21:26.785668 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 31 12:21:26.785668 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application conn.query(q) Oct 31 12:21:26.785668 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 31 12:21:26.785668 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 31 12:21:26.785668 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 31 12:21:26.785668 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application result.read() Oct 31 12:21:26.785668 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 31 12:21:26.785668 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 31 12:21:26.785668 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 31 12:21:26.785668 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 31 12:21:26.788106 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 31 12:21:26.788106 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 31 12:21:26.788106 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 31 12:21:26.788106 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 31 12:21:26.788106 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 31 12:21:26.788106 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 31 12:21:26.788106 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [parameters: {'user_id': 'd924aea0f5b04436a86ee73ad0ed4c1d', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 31 12:21:26.788106 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 31 12:21:26.788106 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:21:26.788106 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 31 12:21:26.788106 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:21:26.788106 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:26.788106 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:21:26.788106 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:21:26.788106 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:21:26.790213 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:21:26.790213 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:21:26.790213 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:21:26.790213 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:21:26.790213 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:21:26.790213 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:21:26.790213 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:21:26.790213 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Oct 31 12:21:26.790213 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Oct 31 12:21:26.790213 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:21:26.790213 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:21:26.790213 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 31 12:21:26.790213 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 31 12:21:26.790213 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 31 12:21:26.790213 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 31 12:21:26.790213 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Oct 31 12:21:26.790213 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Oct 31 12:21:26.792642 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Oct 31 12:21:26.792642 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Oct 31 12:21:26.792642 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 31 12:21:26.792642 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 31 12:21:26.792642 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 31 12:21:26.792642 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:21:26.792642 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 374/760] 10.222.0.15 () {60 vars in 1131 bytes} [Mon Oct 31 12:21:26 2022] POST /identity/v3/users => generated 160 bytes in 182 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 31 12:21:26.793581 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54ac219-a6fa-40a0-bd14-e0ee2b716189 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:26.801924 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451c0834-f065-4d34-bb4c-423afe817af5 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:26.847730 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:26.898245 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54ac219-a6fa-40a0-bd14-e0ee2b716189 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:26.910996 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451c0834-f065-4d34-bb4c-423afe817af5 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:26.936579 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f89a8b-ed7d-4cb5-883e-7365bef0d05f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:26.937200 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f89a8b-ed7d-4cb5-883e-7365bef0d05f None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:26.937475 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f89a8b-ed7d-4cb5-883e-7365bef0d05f None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:26.937742 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f89a8b-ed7d-4cb5-883e-7365bef0d05f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:26.944671 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f89a8b-ed7d-4cb5-883e-7365bef0d05f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:26.946078 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f89a8b-ed7d-4cb5-883e-7365bef0d05f None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:26.948661 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beab7259-c354-4abb-828f-ce408d745e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:26.949758 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beab7259-c354-4abb-828f-ce408d745e8c None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:26.950038 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beab7259-c354-4abb-828f-ce408d745e8c None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:26.950307 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beab7259-c354-4abb-828f-ce408d745e8c None admin] PATH_INFO: `/v3/services/image` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:26.950891 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beab7259-c354-4abb-828f-ce408d745e8c None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:26.952474 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beab7259-c354-4abb-828f-ce408d745e8c None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:26.958021 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-23f89a8b-ed7d-4cb5-883e-7365bef0d05f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 31 12:21:26.958021 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:26.958021 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:21:26.958021 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:21:26.958021 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:21:26.958021 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:21:26.958021 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:21:26.958021 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:21:26.958021 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:21:26.958021 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:21:26.958021 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:21:26.958021 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:21:26.958021 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 31 12:21:26.958021 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 31 12:21:26.958021 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 31 12:21:26.958021 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 31 12:21:26.958926 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:21:26.958926 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:21:26.958926 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 31 12:21:26.958926 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 31 12:21:26.958926 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 31 12:21:26.958926 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 31 12:21:26.958926 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:21:26.958926 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:21:26.958926 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:21:26.958926 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:21:26.958926 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:21:26.958926 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 31 12:21:26.958926 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:21:26.958926 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:21:26.958926 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:21:26.958926 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:21:26.958926 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:21:26.958926 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:21:26.959984 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:21:26.959984 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:21:26.959984 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 31 12:21:26.959984 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 31 12:21:26.959984 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 31 12:21:26.959984 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 31 12:21:26.959984 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 31 12:21:26.959984 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 31 12:21:26.959984 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 31 12:21:26.959984 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:21:26.959984 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 387/761] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Oct 31 12:21:26 2022] GET /identity/v3/users/cinder => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:21:26.960586 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f89a8b-ed7d-4cb5-883e-7365bef0d05f None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:26.964037 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-beab7259-c354-4abb-828f-ce408d745e8c None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 31 12:21:26.964037 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:26.964037 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 31 12:21:26.964037 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 31 12:21:26.964037 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 31 12:21:26.964037 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 31 12:21:26.964037 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 31 12:21:26.964037 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 31 12:21:26.964037 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 31 12:21:26.964037 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:21:26.964037 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 31 12:21:26.964037 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:21:26.964037 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:26.964037 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:21:26.964037 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:21:26.964037 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:21:26.964037 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:21:26.965087 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:21:26.965087 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:21:26.965087 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:21:26.965087 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:21:26.965087 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:21:26.965087 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:21:26.965087 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 31 12:21:26.965087 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 31 12:21:26.965087 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 31 12:21:26.965087 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 31 12:21:26.965087 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:21:26.965087 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:21:26.965087 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:21:26.965087 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:21:26.965087 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:21:26.965087 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:21:26.965911 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:21:26.965911 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:21:26.965911 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:21:26.965911 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:21:26.965911 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:21:26.965911 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:21:26.965911 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:21:26.965911 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:21:26.965911 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:21:26.965911 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:21:26.965911 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 31 12:21:26.965911 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 31 12:21:26.965911 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 31 12:21:26.965911 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:21:26.965911 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 375/762] 10.222.0.15 () {56 vars in 1105 bytes} [Mon Oct 31 12:21:26 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 31 12:21:26.966718 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beab7259-c354-4abb-828f-ce408d745e8c None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:27.002940 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:27.067058 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:27.073859 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f89a8b-ed7d-4cb5-883e-7365bef0d05f None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:27.080844 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beab7259-c354-4abb-828f-ce408d745e8c None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:27.115070 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bd5f24-6cc7-444a-ac01-d29b86bb56aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:27.115824 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c15de4-50c4-432f-89ef-7cda672c3c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:27.116428 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c15de4-50c4-432f-89ef-7cda672c3c32 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:27.116705 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c15de4-50c4-432f-89ef-7cda672c3c32 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:27.116974 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c15de4-50c4-432f-89ef-7cda672c3c32 None admin] PATH_INFO: `/v3/users` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:27.117659 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c15de4-50c4-432f-89ef-7cda672c3c32 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:27.117955 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c15de4-50c4-432f-89ef-7cda672c3c32 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:27.119798 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bd5f24-6cc7-444a-ac01-d29b86bb56aa None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:27.120034 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c15de4-50c4-432f-89ef-7cda672c3c32 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:27.120154 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bd5f24-6cc7-444a-ac01-d29b86bb56aa None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:27.120338 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bd5f24-6cc7-444a-ac01-d29b86bb56aa None admin] PATH_INFO: `/v3/services/compute` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:27.120920 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bd5f24-6cc7-444a-ac01-d29b86bb56aa None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:27.122469 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bd5f24-6cc7-444a-ac01-d29b86bb56aa None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:27.132485 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-28bd5f24-6cc7-444a-ac01-d29b86bb56aa None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 31 12:21:27.132485 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:27.132485 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 31 12:21:27.132485 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 31 12:21:27.132485 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 31 12:21:27.132485 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 31 12:21:27.132485 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 31 12:21:27.132485 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 31 12:21:27.132485 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 31 12:21:27.132485 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:21:27.132485 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 31 12:21:27.132485 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:21:27.132485 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:27.132485 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:21:27.132485 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:21:27.132485 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:21:27.132485 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:21:27.133460 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:21:27.133460 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:21:27.133460 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:21:27.133460 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:21:27.133460 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:21:27.133460 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:21:27.133460 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 31 12:21:27.133460 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 31 12:21:27.133460 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 31 12:21:27.133460 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 31 12:21:27.133460 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:21:27.133460 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:21:27.133460 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:21:27.133460 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:21:27.133460 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:21:27.133460 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:21:27.134374 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:21:27.134374 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 31 12:21:27.134374 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:21:27.134374 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:21:27.134374 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:21:27.134374 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:21:27.134374 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:21:27.134374 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:21:27.134374 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:21:27.134374 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:21:27.134374 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 31 12:21:27.134374 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 31 12:21:27.134374 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 31 12:21:27.134374 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:21:27.134374 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 388/763] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Oct 31 12:21:26 2022] GET /identity/v3/services/compute => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:21:27.139747 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bd5f24-6cc7-444a-ac01-d29b86bb56aa None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:27.161905 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 376/764] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Oct 31 12:21:26 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:21:27.162747 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c15de4-50c4-432f-89ef-7cda672c3c32 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:27.273808 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:27.277518 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bd5f24-6cc7-444a-ac01-d29b86bb56aa None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:27.304097 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c15de4-50c4-432f-89ef-7cda672c3c32 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:27.317520 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc44eb0c-0e4a-4110-aa9a-146474de4c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:27.318140 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc44eb0c-0e4a-4110-aa9a-146474de4c64 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:27.318420 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc44eb0c-0e4a-4110-aa9a-146474de4c64 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:27.318708 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc44eb0c-0e4a-4110-aa9a-146474de4c64 None admin] PATH_INFO: `/v3/services` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:27.319382 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc44eb0c-0e4a-4110-aa9a-146474de4c64 None admin] RBAC: Adding query filter params (name=image) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:27.319692 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc44eb0c-0e4a-4110-aa9a-146474de4c64 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:27.321090 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc44eb0c-0e4a-4110-aa9a-146474de4c64 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:27.327026 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 389/765] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Oct 31 12:21:27 2022] GET /identity/v3/services?name=image => generated 122 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:21:27.336215 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc44eb0c-0e4a-4110-aa9a-146474de4c64 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:27.346885 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c559f317-2d41-4a3d-8064-caed2c9c0270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:27.347765 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c559f317-2d41-4a3d-8064-caed2c9c0270 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:27.348059 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c559f317-2d41-4a3d-8064-caed2c9c0270 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:27.348315 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c559f317-2d41-4a3d-8064-caed2c9c0270 None admin] PATH_INFO: `/v3/services` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:27.348946 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c559f317-2d41-4a3d-8064-caed2c9c0270 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:27.349247 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c559f317-2d41-4a3d-8064-caed2c9c0270 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:27.350614 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c559f317-2d41-4a3d-8064-caed2c9c0270 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:27.357380 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 377/766] 10.222.0.15 () {56 vars in 1112 bytes} [Mon Oct 31 12:21:27 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:21:27.364977 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c559f317-2d41-4a3d-8064-caed2c9c0270 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:27.378741 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:27.463716 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc44eb0c-0e4a-4110-aa9a-146474de4c64 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:27.487809 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c559f317-2d41-4a3d-8064-caed2c9c0270 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:27.510795 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a64f447-70ef-4523-8846-96f5061300c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:27.512103 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a64f447-70ef-4523-8846-96f5061300c7 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:27.512103 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a64f447-70ef-4523-8846-96f5061300c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:27.512103 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a64f447-70ef-4523-8846-96f5061300c7 None admin] PATH_INFO: `/v3/services` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:27.512940 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a64f447-70ef-4523-8846-96f5061300c7 None admin] RBAC: Adding query filter params (type=image) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:27.513048 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a64f447-70ef-4523-8846-96f5061300c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:27.514574 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a64f447-70ef-4523-8846-96f5061300c7 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:27.523337 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 390/767] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Oct 31 12:21:27 2022] GET /identity/v3/services?type=image => generated 350 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 31 12:21:27.529943 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ab8855-4c81-47b4-8e31-8fa3a4469884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:27.530306 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a64f447-70ef-4523-8846-96f5061300c7 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:27.530560 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ab8855-4c81-47b4-8e31-8fa3a4469884 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:27.530867 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ab8855-4c81-47b4-8e31-8fa3a4469884 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:27.531205 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ab8855-4c81-47b4-8e31-8fa3a4469884 None admin] PATH_INFO: `/v3/services` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:27.531777 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ab8855-4c81-47b4-8e31-8fa3a4469884 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:27.532079 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ab8855-4c81-47b4-8e31-8fa3a4469884 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:27.533669 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ab8855-4c81-47b4-8e31-8fa3a4469884 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:27.540589 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 378/768] 10.222.0.15 () {56 vars in 1112 bytes} [Mon Oct 31 12:21:27 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:21:27.546322 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ab8855-4c81-47b4-8e31-8fa3a4469884 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:27.564577 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:27.625808 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:27.646214 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a64f447-70ef-4523-8846-96f5061300c7 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:27.653552 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ab8855-4c81-47b4-8e31-8fa3a4469884 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:27.684584 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d268fcd6-0102-4a5c-919c-741260580d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:27.685298 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d268fcd6-0102-4a5c-919c-741260580d52 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:27.685580 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d268fcd6-0102-4a5c-919c-741260580d52 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:27.685849 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d268fcd6-0102-4a5c-919c-741260580d52 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:27.686546 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d268fcd6-0102-4a5c-919c-741260580d52 None admin] RBAC: Adding query filter params (interface=public, service_id=7dc21b702c2045e6a62a160484e89324, region_id=RegionOne) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:27.686860 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d268fcd6-0102-4a5c-919c-741260580d52 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:27.688251 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d268fcd6-0102-4a5c-919c-741260580d52 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:27.691763 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bc8020-da80-4d90-94d5-863dcc854e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:27.692365 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bc8020-da80-4d90-94d5-863dcc854e9c None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:27.692640 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bc8020-da80-4d90-94d5-863dcc854e9c None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:27.692905 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bc8020-da80-4d90-94d5-863dcc854e9c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:27.693605 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bc8020-da80-4d90-94d5-863dcc854e9c None admin] RBAC: Adding query filter params (interface=public, service_id=13c62fdc89974f2ab0aef996744ff74d, region_id=RegionOne) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:27.698428 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bc8020-da80-4d90-94d5-863dcc854e9c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:27.698564 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 391/769] 10.222.0.15 () {56 vars in 1251 bytes} [Mon Oct 31 12:21:27 2022] GET /identity/v3/endpoints?service_id=7dc21b702c2045e6a62a160484e89324&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:21:27.698643 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bc8020-da80-4d90-94d5-863dcc854e9c None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:27.699419 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d268fcd6-0102-4a5c-919c-741260580d52 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:27.703627 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 379/770] 10.222.0.15 () {56 vars in 1251 bytes} [Mon Oct 31 12:21:27 2022] GET /identity/v3/endpoints?service_id=13c62fdc89974f2ab0aef996744ff74d&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 31 12:21:27.708851 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bc8020-da80-4d90-94d5-863dcc854e9c None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:27.798117 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:27.809244 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d268fcd6-0102-4a5c-919c-741260580d52 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:27.815819 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bc8020-da80-4d90-94d5-863dcc854e9c None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:27.845513 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4308f739-00e0-4f03-b918-4e47f3e2aa89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:27.846340 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4308f739-00e0-4f03-b918-4e47f3e2aa89 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:27.846720 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4308f739-00e0-4f03-b918-4e47f3e2aa89 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:27.847088 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4308f739-00e0-4f03-b918-4e47f3e2aa89 None admin] PATH_INFO: `/v3/services` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:27.847818 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4308f739-00e0-4f03-b918-4e47f3e2aa89 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:27.849394 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4308f739-00e0-4f03-b918-4e47f3e2aa89 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:27.852307 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249649c5-1005-4c9c-853d-5afb218ab118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:27.853504 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249649c5-1005-4c9c-853d-5afb218ab118 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:27.853829 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249649c5-1005-4c9c-853d-5afb218ab118 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:27.854183 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249649c5-1005-4c9c-853d-5afb218ab118 None admin] PATH_INFO: `/v3/services` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:27.854784 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249649c5-1005-4c9c-853d-5afb218ab118 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:27.856160 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249649c5-1005-4c9c-853d-5afb218ab118 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:27.871203 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 392/771] 10.222.0.15 () {56 vars in 1087 bytes} [Mon Oct 31 12:21:27 2022] GET /identity/v3/services => generated 1957 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:21:27.871419 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 380/772] 10.222.0.15 () {56 vars in 1087 bytes} [Mon Oct 31 12:21:27 2022] GET /identity/v3/services => generated 1957 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 31 12:21:28.104731 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228b24ea-d01e-4dcf-8b7f-647ec241f58b None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:28.105115 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228b24ea-d01e-4dcf-8b7f-647ec241f58b None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:28.105462 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228b24ea-d01e-4dcf-8b7f-647ec241f58b None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:28.106886 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 393/773] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:21:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 31 12:21:28.112515 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3980b273-7a94-4e89-bab0-217a3993bc0d None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:28.112795 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3980b273-7a94-4e89-bab0-217a3993bc0d None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:28.113066 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3980b273-7a94-4e89-bab0-217a3993bc0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:28.120189 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:28.133369 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-3980b273-7a94-4e89-bab0-217a3993bc0d None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:21:28.135800 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-3980b273-7a94-4e89-bab0-217a3993bc0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:21:28.205617 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 381/774] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:21:28 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 31 12:21:28.474178 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228b24ea-d01e-4dcf-8b7f-647ec241f58b None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:28.576356 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228b24ea-d01e-4dcf-8b7f-647ec241f58b None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:28.611425 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f20ee6-0bcf-4c78-9375-9e9460bda2e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:28.612058 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f20ee6-0bcf-4c78-9375-9e9460bda2e6 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:28.612333 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f20ee6-0bcf-4c78-9375-9e9460bda2e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:28.612602 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f20ee6-0bcf-4c78-9375-9e9460bda2e6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:28.613709 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f20ee6-0bcf-4c78-9375-9e9460bda2e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:28.615116 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f20ee6-0bcf-4c78-9375-9e9460bda2e6 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:28.616869 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 394/775] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Oct 31 12:21:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:21:28.622612 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3980b273-7a94-4e89-bab0-217a3993bc0d None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:28.726413 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3980b273-7a94-4e89-bab0-217a3993bc0d None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:28.755421 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:28.762451 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ccd10a-77cc-478d-9efd-a9240c2f9d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:28.763062 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ccd10a-77cc-478d-9efd-a9240c2f9d93 None admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:28.763366 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ccd10a-77cc-478d-9efd-a9240c2f9d93 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:28.763638 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ccd10a-77cc-478d-9efd-a9240c2f9d93 None admin] PATH_INFO: `/v3/projects` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:28.764293 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ccd10a-77cc-478d-9efd-a9240c2f9d93 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:28.765697 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ccd10a-77cc-478d-9efd-a9240c2f9d93 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:28.799615 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 382/776] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Oct 31 12:21:28 2022] POST /identity/v3/projects => generated 311 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:21:30.060985 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc03c134-2f28-4651-9aa9-389422a6d385 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:30.061564 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc03c134-2f28-4651-9aa9-389422a6d385 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:30.061564 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc03c134-2f28-4651-9aa9-389422a6d385 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:30.063118 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 395/777] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:21:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:21:30.068729 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44633a1a-6ea0-4dd5-a944-1a7df3621c9f None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:30.069225 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44633a1a-6ea0-4dd5-a944-1a7df3621c9f None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:30.069327 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44633a1a-6ea0-4dd5-a944-1a7df3621c9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:30.087733 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-44633a1a-6ea0-4dd5-a944-1a7df3621c9f None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:21:30.089062 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-44633a1a-6ea0-4dd5-a944-1a7df3621c9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:21:30.157362 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 383/778] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:21:30 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 31 12:21:30.168173 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e731ed9-4c5f-4290-ac7e-9d935903e6a9 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:30.168460 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e731ed9-4c5f-4290-ac7e-9d935903e6a9 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:30.168733 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e731ed9-4c5f-4290-ac7e-9d935903e6a9 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:30.170184 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 396/779] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:21:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:21:30.175223 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66765568-8d1f-4c19-9a86-ffe97dbd1b2b None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:30.175521 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66765568-8d1f-4c19-9a86-ffe97dbd1b2b None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:30.175808 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66765568-8d1f-4c19-9a86-ffe97dbd1b2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:30.194502 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-66765568-8d1f-4c19-9a86-ffe97dbd1b2b None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:21:30.195699 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-66765568-8d1f-4c19-9a86-ffe97dbd1b2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:21:30.265129 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 384/780] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:21:30 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 31 12:21:30.421721 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e731ed9-4c5f-4290-ac7e-9d935903e6a9 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:30.446118 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:30.523387 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e731ed9-4c5f-4290-ac7e-9d935903e6a9 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:30.544835 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66765568-8d1f-4c19-9a86-ffe97dbd1b2b None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:30.558799 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b28ee80-758b-426e-813e-08d0bd1af7fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:30.559459 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b28ee80-758b-426e-813e-08d0bd1af7fc None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:30.559733 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b28ee80-758b-426e-813e-08d0bd1af7fc None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:30.559997 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b28ee80-758b-426e-813e-08d0bd1af7fc None admin] PATH_INFO: `/v3/services/image` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:30.560569 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b28ee80-758b-426e-813e-08d0bd1af7fc None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:30.561911 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b28ee80-758b-426e-813e-08d0bd1af7fc None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:30.571086 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-8b28ee80-758b-426e-813e-08d0bd1af7fc None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 31 12:21:30.571086 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:30.571086 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 31 12:21:30.571086 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 31 12:21:30.571086 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 31 12:21:30.571086 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 31 12:21:30.571086 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 31 12:21:30.571086 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 31 12:21:30.571086 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 31 12:21:30.571086 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:21:30.571086 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 31 12:21:30.571086 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:21:30.571086 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:30.571086 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:21:30.571086 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:21:30.571086 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:21:30.571086 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:21:30.573598 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:21:30.573598 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:21:30.573598 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:21:30.573598 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:21:30.573598 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:21:30.573598 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:21:30.573598 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 31 12:21:30.573598 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 31 12:21:30.573598 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 31 12:21:30.573598 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 31 12:21:30.573598 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:21:30.573598 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:21:30.573598 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:21:30.573598 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:21:30.573598 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:21:30.573598 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:21:30.574735 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:21:30.574735 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 31 12:21:30.574735 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:21:30.574735 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:21:30.574735 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:21:30.574735 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:21:30.574735 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:21:30.574735 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:21:30.574735 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:21:30.574735 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:21:30.574735 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 31 12:21:30.574735 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 31 12:21:30.574735 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 31 12:21:30.574735 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:21:30.574735 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 397/781] 10.222.0.15 () {56 vars in 1105 bytes} [Mon Oct 31 12:21:30 2022] GET /identity/v3/services/image => generated 86 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:21:30.578594 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b28ee80-758b-426e-813e-08d0bd1af7fc None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:30.598559 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:30.660705 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66765568-8d1f-4c19-9a86-ffe97dbd1b2b None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:30.677172 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:30.692796 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b28ee80-758b-426e-813e-08d0bd1af7fc None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:30.698872 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fe94db-7768-4b11-8ae5-5b1035f97480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:30.699734 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fe94db-7768-4b11-8ae5-5b1035f97480 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:30.700185 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fe94db-7768-4b11-8ae5-5b1035f97480 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:30.700639 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fe94db-7768-4b11-8ae5-5b1035f97480 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:30.701218 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fe94db-7768-4b11-8ae5-5b1035f97480 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:30.702621 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fe94db-7768-4b11-8ae5-5b1035f97480 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:30.711506 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-04fe94db-7768-4b11-8ae5-5b1035f97480 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 31 12:21:30.711506 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:30.711506 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 31 12:21:30.711506 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 31 12:21:30.711506 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 31 12:21:30.711506 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 31 12:21:30.711506 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 31 12:21:30.711506 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 31 12:21:30.711506 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 31 12:21:30.711506 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:21:30.711506 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 31 12:21:30.711506 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:21:30.711506 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:30.711506 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:21:30.711506 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:21:30.711506 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:21:30.711506 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:21:30.715390 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:21:30.715390 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:21:30.715390 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:21:30.715390 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:21:30.715390 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:21:30.715390 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:21:30.715390 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 31 12:21:30.715390 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 31 12:21:30.715390 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 31 12:21:30.715390 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 31 12:21:30.715390 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:21:30.715390 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:21:30.715390 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:21:30.715390 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:21:30.715390 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:21:30.715390 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:21:30.716365 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:21:30.716365 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:21:30.716365 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:21:30.716365 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:21:30.716365 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:21:30.716365 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:21:30.716365 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:21:30.716365 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:21:30.716365 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:21:30.716365 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:21:30.716365 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 31 12:21:30.716365 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 31 12:21:30.716365 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 31 12:21:30.716365 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:21:30.716365 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 385/782] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Oct 31 12:21:30 2022] GET /identity/v3/services/compute => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 31 12:21:30.718669 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fe94db-7768-4b11-8ae5-5b1035f97480 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:30.733417 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f6f06a-7264-45cc-b2dd-ced3384fc4c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:30.734026 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f6f06a-7264-45cc-b2dd-ced3384fc4c4 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:30.734449 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f6f06a-7264-45cc-b2dd-ced3384fc4c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:30.734737 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f6f06a-7264-45cc-b2dd-ced3384fc4c4 None admin] PATH_INFO: `/v3/services` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:30.735761 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f6f06a-7264-45cc-b2dd-ced3384fc4c4 None admin] RBAC: Adding query filter params (name=image) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:30.736065 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f6f06a-7264-45cc-b2dd-ced3384fc4c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:30.737558 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f6f06a-7264-45cc-b2dd-ced3384fc4c4 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:30.747595 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 398/783] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Oct 31 12:21:30 2022] GET /identity/v3/services?name=image => generated 122 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 31 12:21:30.748605 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f6f06a-7264-45cc-b2dd-ced3384fc4c4 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:30.823987 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fe94db-7768-4b11-8ae5-5b1035f97480 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:30.833304 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:30.856236 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f6f06a-7264-45cc-b2dd-ced3384fc4c4 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:30.860615 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd488d6-a585-4995-8d63-1e5b727350b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:30.861219 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd488d6-a585-4995-8d63-1e5b727350b0 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:30.861499 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd488d6-a585-4995-8d63-1e5b727350b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:30.861765 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd488d6-a585-4995-8d63-1e5b727350b0 None admin] PATH_INFO: `/v3/services` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:30.862380 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd488d6-a585-4995-8d63-1e5b727350b0 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:30.862676 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd488d6-a585-4995-8d63-1e5b727350b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:30.864046 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd488d6-a585-4995-8d63-1e5b727350b0 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:30.869350 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 386/784] 10.222.0.15 () {56 vars in 1112 bytes} [Mon Oct 31 12:21:30 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 31 12:21:30.875024 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd488d6-a585-4995-8d63-1e5b727350b0 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:30.895443 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a03162-3e6e-4a44-8a3f-a63754b9bec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:30.896079 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a03162-3e6e-4a44-8a3f-a63754b9bec7 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:30.896361 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a03162-3e6e-4a44-8a3f-a63754b9bec7 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:30.896635 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a03162-3e6e-4a44-8a3f-a63754b9bec7 None admin] PATH_INFO: `/v3/services` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:30.897276 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a03162-3e6e-4a44-8a3f-a63754b9bec7 None admin] RBAC: Adding query filter params (type=image) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:30.897583 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a03162-3e6e-4a44-8a3f-a63754b9bec7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:30.898987 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a03162-3e6e-4a44-8a3f-a63754b9bec7 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:30.905264 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 399/785] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Oct 31 12:21:30 2022] GET /identity/v3/services?type=image => generated 350 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:21:30.911311 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a03162-3e6e-4a44-8a3f-a63754b9bec7 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:30.931966 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:30.990376 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd488d6-a585-4995-8d63-1e5b727350b0 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:31.021355 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a03162-3e6e-4a44-8a3f-a63754b9bec7 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:31.028686 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e15432-d31f-443d-87fb-d8121331b3f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:31.029328 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e15432-d31f-443d-87fb-d8121331b3f2 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:31.029607 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e15432-d31f-443d-87fb-d8121331b3f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:31.029870 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e15432-d31f-443d-87fb-d8121331b3f2 None admin] PATH_INFO: `/v3/services` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:31.030515 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e15432-d31f-443d-87fb-d8121331b3f2 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:31.030812 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e15432-d31f-443d-87fb-d8121331b3f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:31.032224 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e15432-d31f-443d-87fb-d8121331b3f2 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:31.037022 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 387/786] 10.222.0.15 () {56 vars in 1112 bytes} [Mon Oct 31 12:21:30 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 31 12:21:31.038141 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f48ef1-3a18-4129-9f0d-250441e7cb61 None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:31.038464 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f48ef1-3a18-4129-9f0d-250441e7cb61 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:31.038755 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f48ef1-3a18-4129-9f0d-250441e7cb61 None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:31.039878 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 388/787] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:21:31 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:21:31.043243 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f48ef1-3a18-4129-9f0d-250441e7cb61 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:31.058246 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19fe1ae-128e-4faa-bf70-a36220debcc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:31.058856 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19fe1ae-128e-4faa-bf70-a36220debcc2 None admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:31.059134 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19fe1ae-128e-4faa-bf70-a36220debcc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:31.059429 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19fe1ae-128e-4faa-bf70-a36220debcc2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:31.060091 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19fe1ae-128e-4faa-bf70-a36220debcc2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:31.061433 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19fe1ae-128e-4faa-bf70-a36220debcc2 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:31.081399 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:31.091264 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 400/788] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:21:30 2022] POST /identity/v3/endpoints => generated 327 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 31 12:21:31.091264 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861f85d5-f98e-4e07-a427-3558616a02b0 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:31.091264 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861f85d5-f98e-4e07-a427-3558616a02b0 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:31.091264 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861f85d5-f98e-4e07-a427-3558616a02b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:31.114606 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-861f85d5-f98e-4e07-a427-3558616a02b0 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:21:31.116234 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-861f85d5-f98e-4e07-a427-3558616a02b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:21:31.162139 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f48ef1-3a18-4129-9f0d-250441e7cb61 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:31.199909 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 401/789] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:21:31 2022] POST /identity/v3/auth/tokens => generated 1962 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 31 12:21:31.201926 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00651b48-085c-4d02-937f-9151d9a806bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:31.202798 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00651b48-085c-4d02-937f-9151d9a806bf None admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:31.203331 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00651b48-085c-4d02-937f-9151d9a806bf None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:31.203960 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00651b48-085c-4d02-937f-9151d9a806bf None admin] PATH_INFO: `/v3/endpoints` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:31.204868 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00651b48-085c-4d02-937f-9151d9a806bf None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:31.206441 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00651b48-085c-4d02-937f-9151d9a806bf None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:31.229241 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 389/790] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:21:31 2022] POST /identity/v3/endpoints => generated 334 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 31 12:21:31.469611 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861f85d5-f98e-4e07-a427-3558616a02b0 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:31.471764 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:31.575004 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861f85d5-f98e-4e07-a427-3558616a02b0 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:31.616300 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf55b78-6fec-4233-8cbd-e5a7b20460f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:31.616935 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf55b78-6fec-4233-8cbd-e5a7b20460f3 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:31.617213 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf55b78-6fec-4233-8cbd-e5a7b20460f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:31.617485 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf55b78-6fec-4233-8cbd-e5a7b20460f3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:31.618638 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf55b78-6fec-4233-8cbd-e5a7b20460f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:31.620089 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf55b78-6fec-4233-8cbd-e5a7b20460f3 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:31.623189 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 402/791] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Oct 31 12:21:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:21:31.627563 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00651b48-085c-4d02-937f-9151d9a806bf None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:31.720099 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:31.734148 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00651b48-085c-4d02-937f-9151d9a806bf None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:31.769671 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52625cb-0ef3-4cc5-a928-df06a1cc0e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:31.770283 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52625cb-0ef3-4cc5-a928-df06a1cc0e30 None admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:31.770563 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52625cb-0ef3-4cc5-a928-df06a1cc0e30 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:31.770834 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52625cb-0ef3-4cc5-a928-df06a1cc0e30 None admin] PATH_INFO: `/v3/users` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:31.771557 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52625cb-0ef3-4cc5-a928-df06a1cc0e30 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:31.772976 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52625cb-0ef3-4cc5-a928-df06a1cc0e30 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:31.802615 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 390/792] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Oct 31 12:21:31 2022] POST /identity/v3/users => generated 285 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:21:33.288841 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216d1e06-f4d8-4fe8-8773-620f0aa9f17a None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:33.289377 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216d1e06-f4d8-4fe8-8773-620f0aa9f17a None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:33.289377 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216d1e06-f4d8-4fe8-8773-620f0aa9f17a None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:33.291579 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 403/793] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:21:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:21:33.296548 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3331c390-ef01-4f66-ab65-e5a17d158c3e None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:33.296952 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3331c390-ef01-4f66-ab65-e5a17d158c3e None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:33.297113 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3331c390-ef01-4f66-ab65-e5a17d158c3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:33.315436 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-3331c390-ef01-4f66-ab65-e5a17d158c3e None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:21:33.316634 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-3331c390-ef01-4f66-ab65-e5a17d158c3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:21:33.388169 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 391/794] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:21:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 31 12:21:33.489531 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020d9bcc-0267-4fa7-b40f-6c9f398b4353 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:33.489869 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020d9bcc-0267-4fa7-b40f-6c9f398b4353 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:33.490104 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020d9bcc-0267-4fa7-b40f-6c9f398b4353 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:33.491560 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 404/795] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:21:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:21:33.497220 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df96893-750e-49bf-b870-5f211f6320bc None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:33.497509 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df96893-750e-49bf-b870-5f211f6320bc None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:33.497780 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df96893-750e-49bf-b870-5f211f6320bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:33.515655 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-1df96893-750e-49bf-b870-5f211f6320bc None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:21:33.516813 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-1df96893-750e-49bf-b870-5f211f6320bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:21:33.583631 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 392/796] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:21:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 31 12:21:33.654982 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020d9bcc-0267-4fa7-b40f-6c9f398b4353 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:33.723184 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:33.759652 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020d9bcc-0267-4fa7-b40f-6c9f398b4353 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:33.793389 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbcc3a6-ab28-4020-af5c-7cfd93861126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:33.793990 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbcc3a6-ab28-4020-af5c-7cfd93861126 None admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:33.794335 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbcc3a6-ab28-4020-af5c-7cfd93861126 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:33.794604 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbcc3a6-ab28-4020-af5c-7cfd93861126 None admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:33.795287 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbcc3a6-ab28-4020-af5c-7cfd93861126 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:33.796632 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbcc3a6-ab28-4020-af5c-7cfd93861126 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:33.805340 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.sql.core [None req-cdbcc3a6-ab28-4020-af5c-7cfd93861126 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 31 12:21:33.805340 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 31 12:21:33.805340 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [parameters: {'id': 'd042e64f2d7941db8007b4e273f5b026', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 31 12:21:33.805340 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81491) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 31 12:21:33.813678 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-cdbcc3a6-ab28-4020-af5c-7cfd93861126 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 31 12:21:33.813678 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:33.813678 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 31 12:21:33.813678 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 31 12:21:33.813678 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 31 12:21:33.813678 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 31 12:21:33.813678 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 31 12:21:33.813678 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application result = self._query(query) Oct 31 12:21:33.813678 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 31 12:21:33.813678 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application conn.query(q) Oct 31 12:21:33.813678 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 31 12:21:33.813678 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 31 12:21:33.813678 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 31 12:21:33.813678 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application result.read() Oct 31 12:21:33.814565 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 31 12:21:33.814565 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 31 12:21:33.814565 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 31 12:21:33.814565 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 31 12:21:33.814565 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 31 12:21:33.814565 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 31 12:21:33.814565 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 31 12:21:33.814565 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 31 12:21:33.814565 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 31 12:21:33.814565 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:21:33.814565 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 31 12:21:33.814565 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:21:33.814565 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:33.814565 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 31 12:21:33.814565 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 31 12:21:33.814565 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 31 12:21:33.814565 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 31 12:21:33.814565 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 31 12:21:33.815570 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application next(self.gen) Oct 31 12:21:33.815570 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Oct 31 12:21:33.815570 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application yield resource Oct 31 12:21:33.815570 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 31 12:21:33.815570 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application next(self.gen) Oct 31 12:21:33.815570 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Oct 31 12:21:33.815570 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self.session.rollback() Oct 31 12:21:33.815570 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 31 12:21:33.815570 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self.force_reraise() Oct 31 12:21:33.815570 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 31 12:21:33.815570 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise self.value Oct 31 12:21:33.815570 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Oct 31 12:21:33.815570 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 31 12:21:33.815570 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Oct 31 12:21:33.815570 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application session.commit() Oct 31 12:21:33.815570 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 31 12:21:33.815570 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 31 12:21:33.816504 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 31 12:21:33.816504 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self._prepare_impl() Oct 31 12:21:33.816504 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 31 12:21:33.816504 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self.session.flush() Oct 31 12:21:33.816504 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 31 12:21:33.816504 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self._flush(objects) Oct 31 12:21:33.816504 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 31 12:21:33.816504 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 31 12:21:33.816504 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 31 12:21:33.816504 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application compat.raise_( Oct 31 12:21:33.816504 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 31 12:21:33.816504 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception Oct 31 12:21:33.816504 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 31 12:21:33.816504 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application flush_context.execute() Oct 31 12:21:33.816504 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 31 12:21:33.816504 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rec.execute(self) Oct 31 12:21:33.816504 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 31 12:21:33.817442 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 31 12:21:33.817442 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 31 12:21:33.817442 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 31 12:21:33.817442 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 31 12:21:33.817442 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 31 12:21:33.817442 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 31 12:21:33.817442 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 31 12:21:33.817442 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 31 12:21:33.817442 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 31 12:21:33.817442 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 31 12:21:33.817442 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 31 12:21:33.817442 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 31 12:21:33.817442 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 31 12:21:33.817442 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 31 12:21:33.817442 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 31 12:21:33.818341 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 31 12:21:33.818341 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception Oct 31 12:21:33.818341 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 31 12:21:33.818341 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 31 12:21:33.818341 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 31 12:21:33.818341 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 31 12:21:33.818341 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 31 12:21:33.818341 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application result = self._query(query) Oct 31 12:21:33.818341 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 31 12:21:33.818341 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application conn.query(q) Oct 31 12:21:33.818341 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 31 12:21:33.818341 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 31 12:21:33.818341 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 31 12:21:33.818341 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application result.read() Oct 31 12:21:33.818341 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 31 12:21:33.818341 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 31 12:21:33.818341 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 31 12:21:33.820699 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 31 12:21:33.820699 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 31 12:21:33.820699 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 31 12:21:33.820699 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 31 12:21:33.820699 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 31 12:21:33.820699 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 31 12:21:33.820699 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 31 12:21:33.820699 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [parameters: {'id': 'd042e64f2d7941db8007b4e273f5b026', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 31 12:21:33.820699 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 31 12:21:33.820699 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:21:33.820699 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 31 12:21:33.820699 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:21:33.820699 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:33.820699 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 31 12:21:33.820699 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 31 12:21:33.822772 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 31 12:21:33.822772 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 31 12:21:33.822772 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 31 12:21:33.822772 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:21:33.822772 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 31 12:21:33.822772 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:21:33.822772 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:33.822772 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:21:33.822772 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:21:33.822772 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:21:33.822772 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:21:33.822772 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:21:33.822772 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:21:33.822772 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:21:33.822772 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:21:33.822772 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:21:33.822772 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:21:33.826850 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 31 12:21:33.826850 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 31 12:21:33.826850 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:21:33.826850 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:21:33.826850 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 31 12:21:33.826850 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 31 12:21:33.826850 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:21:33.826850 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:21:33.826850 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 31 12:21:33.826850 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 31 12:21:33.826850 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 31 12:21:33.826850 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:21:33.826850 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 405/797] 10.222.0.15 () {60 vars in 1137 bytes} [Mon Oct 31 12:21:33 2022] POST /identity/v3/domains => generated 190 bytes in 160 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:21:33.828991 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df96893-750e-49bf-b870-5f211f6320bc None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:33.830768 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:33.855539 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbcc3a6-ab28-4020-af5c-7cfd93861126 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:33.940446 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df96893-750e-49bf-b870-5f211f6320bc None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:33.965780 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbcc3a6-ab28-4020-af5c-7cfd93861126 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:33.975504 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c22753-646d-4159-9d48-909f4e784005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:33.976134 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c22753-646d-4159-9d48-909f4e784005 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:33.976410 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c22753-646d-4159-9d48-909f4e784005 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:33.976674 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c22753-646d-4159-9d48-909f4e784005 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:33.987278 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c22753-646d-4159-9d48-909f4e784005 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:33.988702 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c22753-646d-4159-9d48-909f4e784005 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:33.998354 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:34.003688 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-e2c22753-646d-4159-9d48-909f4e784005 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:21:34.003688 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:34.003688 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:21:34.003688 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:21:34.003688 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:21:34.003688 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:21:34.003688 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:21:34.003688 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:21:34.003688 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:21:34.003688 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:21:34.003688 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:21:34.003688 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:21:34.003688 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 31 12:21:34.003688 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 31 12:21:34.003688 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 31 12:21:34.003688 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 31 12:21:34.004866 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:21:34.004866 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:21:34.004866 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:21:34.004866 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:21:34.004866 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:21:34.004866 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:21:34.004866 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:21:34.004866 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:21:34.004866 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:21:34.004866 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:21:34.004866 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:21:34.004866 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:21:34.004866 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:21:34.004866 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:21:34.004866 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:21:34.004866 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:21:34.004866 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 31 12:21:34.006028 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 31 12:21:34.006028 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:21:34.006028 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:21:34.006028 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 393/798] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Oct 31 12:21:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:21:34.006028 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1432b56b-417c-4f4b-b655-757d07b3bcda None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:34.006409 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1432b56b-417c-4f4b-b655-757d07b3bcda None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:34.006532 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1432b56b-417c-4f4b-b655-757d07b3bcda None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:34.007911 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 394/799] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:21:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:21:34.011147 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39684931-c42e-463d-8428-b0ad0a79cfd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:34.011308 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1432b56b-417c-4f4b-b655-757d07b3bcda None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:34.011812 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39684931-c42e-463d-8428-b0ad0a79cfd5 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:34.012249 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39684931-c42e-463d-8428-b0ad0a79cfd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:34.012468 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39684931-c42e-463d-8428-b0ad0a79cfd5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:34.026501 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39684931-c42e-463d-8428-b0ad0a79cfd5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 31 12:21:34.031308 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-39684931-c42e-463d-8428-b0ad0a79cfd5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 31 12:21:34.031308 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:34.031308 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:21:34.031308 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:21:34.031308 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:21:34.031308 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:21:34.031308 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:21:34.031308 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:21:34.031308 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:21:34.031308 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:21:34.031308 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:21:34.031308 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:21:34.031308 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 31 12:21:34.031308 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 31 12:21:34.031308 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 31 12:21:34.031308 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 31 12:21:34.032319 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 31 12:21:34.032319 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 31 12:21:34.032319 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 31 12:21:34.032319 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 31 12:21:34.032319 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:21:34.032319 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:21:34.032319 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:21:34.032319 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:21:34.032319 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:21:34.032319 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:21:34.032319 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:21:34.032319 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 31 12:21:34.032319 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:21:34.032319 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:21:34.032319 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:21:34.032319 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:21:34.033156 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:21:34.033156 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:21:34.033156 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:21:34.033156 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:21:34.033156 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 31 12:21:34.033156 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 31 12:21:34.033156 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 31 12:21:34.033156 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 31 12:21:34.033156 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 31 12:21:34.033156 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 31 12:21:34.033156 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 31 12:21:34.033156 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:21:34.033156 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 406/800] 10.222.0.15 () {56 vars in 1120 bytes} [Mon Oct 31 12:21:33 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 31 12:21:34.034182 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e827b6c-503f-4cba-bf28-99c9333a2b22 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:34.034461 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e827b6c-503f-4cba-bf28-99c9333a2b22 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:34.034731 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e827b6c-503f-4cba-bf28-99c9333a2b22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:34.051424 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-2e827b6c-503f-4cba-bf28-99c9333a2b22 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:21:34.052585 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-2e827b6c-503f-4cba-bf28-99c9333a2b22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:21:34.076119 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:34.118450 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 407/801] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:21:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 31 12:21:34.119232 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e827b6c-503f-4cba-bf28-99c9333a2b22 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:34.123916 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1432b56b-417c-4f4b-b655-757d07b3bcda None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:34.162600 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b33e9fa-26ff-45e9-9858-d2a51c9a7065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:34.163718 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b33e9fa-26ff-45e9-9858-d2a51c9a7065 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:34.163997 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b33e9fa-26ff-45e9-9858-d2a51c9a7065 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:34.164266 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b33e9fa-26ff-45e9-9858-d2a51c9a7065 None admin] PATH_INFO: `/v3/domains` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:34.165828 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b33e9fa-26ff-45e9-9858-d2a51c9a7065 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:34.166275 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b33e9fa-26ff-45e9-9858-d2a51c9a7065 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:34.168000 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b33e9fa-26ff-45e9-9858-d2a51c9a7065 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:34.180668 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 395/802] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Oct 31 12:21:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 31 12:21:34.230759 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e827b6c-503f-4cba-bf28-99c9333a2b22 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:34.254579 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:34.271037 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c989b70-30e2-42f0-94a2-89e7940bbfee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:34.271689 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c989b70-30e2-42f0-94a2-89e7940bbfee None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:34.271967 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c989b70-30e2-42f0-94a2-89e7940bbfee None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:34.272295 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c989b70-30e2-42f0-94a2-89e7940bbfee None admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:34.272974 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c989b70-30e2-42f0-94a2-89e7940bbfee None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:34.273277 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c989b70-30e2-42f0-94a2-89e7940bbfee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:34.275204 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c989b70-30e2-42f0-94a2-89e7940bbfee None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:34.289164 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 408/803] 10.222.0.15 () {56 vars in 1115 bytes} [Mon Oct 31 12:21:34 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:21:34.290521 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b33e9fa-26ff-45e9-9858-d2a51c9a7065 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:34.395536 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c989b70-30e2-42f0-94a2-89e7940bbfee None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:34.398951 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b33e9fa-26ff-45e9-9858-d2a51c9a7065 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:34.416429 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:34.436524 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc790ca-54a2-45cf-ae2b-074b6facd6d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:34.437151 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc790ca-54a2-45cf-ae2b-074b6facd6d4 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:34.437425 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc790ca-54a2-45cf-ae2b-074b6facd6d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:34.437692 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc790ca-54a2-45cf-ae2b-074b6facd6d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:34.452752 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc790ca-54a2-45cf-ae2b-074b6facd6d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:34.453384 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc790ca-54a2-45cf-ae2b-074b6facd6d4 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:34.470167 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-dcc790ca-54a2-45cf-ae2b-074b6facd6d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:21:34.470167 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:34.470167 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:21:34.470167 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:21:34.470167 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:21:34.470167 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:21:34.470167 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:21:34.470167 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:21:34.470167 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:21:34.470167 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:21:34.470167 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:21:34.470167 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:21:34.470167 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 31 12:21:34.470167 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 31 12:21:34.470167 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 31 12:21:34.470167 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 31 12:21:34.471450 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:21:34.471450 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:21:34.471450 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:21:34.471450 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:21:34.471450 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:21:34.471450 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:21:34.471450 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:21:34.471450 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:21:34.471450 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:21:34.471450 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:21:34.471450 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:21:34.471450 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:21:34.471450 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:21:34.471450 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:21:34.471450 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:21:34.471450 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:21:34.471450 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 31 12:21:34.472527 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 31 12:21:34.472527 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:21:34.472527 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:21:34.472527 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 396/804] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Oct 31 12:21:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:21:34.477231 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc790ca-54a2-45cf-ae2b-074b6facd6d4 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:34.520473 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c989b70-30e2-42f0-94a2-89e7940bbfee None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:34.554889 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eacd0c2-556f-4771-9ffa-e7a12f0c8ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:34.555629 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eacd0c2-556f-4771-9ffa-e7a12f0c8ced None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:34.556292 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eacd0c2-556f-4771-9ffa-e7a12f0c8ced None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:34.556637 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eacd0c2-556f-4771-9ffa-e7a12f0c8ced None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:34.566673 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eacd0c2-556f-4771-9ffa-e7a12f0c8ced None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 31 12:21:34.571997 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-6eacd0c2-556f-4771-9ffa-e7a12f0c8ced None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 31 12:21:34.571997 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:34.571997 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:21:34.571997 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:21:34.571997 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:21:34.571997 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:21:34.571997 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:21:34.571997 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:21:34.571997 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:21:34.571997 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:21:34.571997 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:21:34.571997 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:21:34.571997 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 31 12:21:34.571997 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 31 12:21:34.571997 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 31 12:21:34.571997 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 31 12:21:34.573179 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 31 12:21:34.573179 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 31 12:21:34.573179 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 31 12:21:34.573179 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 31 12:21:34.573179 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:21:34.573179 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:21:34.573179 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:21:34.573179 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:21:34.573179 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:21:34.573179 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:21:34.573179 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:21:34.573179 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 31 12:21:34.573179 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:21:34.573179 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:21:34.573179 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:21:34.573179 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:21:34.574215 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:21:34.574215 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:21:34.574215 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:21:34.574215 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:21:34.574215 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 31 12:21:34.574215 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 31 12:21:34.574215 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 31 12:21:34.574215 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 31 12:21:34.574215 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 31 12:21:34.574215 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 31 12:21:34.574215 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 31 12:21:34.574215 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:21:34.574215 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 409/805] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Oct 31 12:21:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 31 12:21:34.579116 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eacd0c2-556f-4771-9ffa-e7a12f0c8ced None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:34.589689 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc790ca-54a2-45cf-ae2b-074b6facd6d4 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:34.621393 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:34.623401 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f30eae8-53b7-4e7e-9c65-41c4b62789e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:34.624090 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f30eae8-53b7-4e7e-9c65-41c4b62789e6 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:34.624382 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f30eae8-53b7-4e7e-9c65-41c4b62789e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:34.624652 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f30eae8-53b7-4e7e-9c65-41c4b62789e6 None admin] PATH_INFO: `/v3/domains` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:34.625271 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f30eae8-53b7-4e7e-9c65-41c4b62789e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:34.625562 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f30eae8-53b7-4e7e-9c65-41c4b62789e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:34.626904 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f30eae8-53b7-4e7e-9c65-41c4b62789e6 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:34.639278 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 397/806] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Oct 31 12:21:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 31 12:21:34.645787 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f30eae8-53b7-4e7e-9c65-41c4b62789e6 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:34.674075 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:34.681946 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eacd0c2-556f-4771-9ffa-e7a12f0c8ced None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:34.716786 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfe7020-f0f9-4018-b04a-d4ce2240963e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:34.717396 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfe7020-f0f9-4018-b04a-d4ce2240963e None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:34.717666 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfe7020-f0f9-4018-b04a-d4ce2240963e None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:34.717929 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfe7020-f0f9-4018-b04a-d4ce2240963e None admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:34.718574 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfe7020-f0f9-4018-b04a-d4ce2240963e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:34.718872 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfe7020-f0f9-4018-b04a-d4ce2240963e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:34.720234 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfe7020-f0f9-4018-b04a-d4ce2240963e None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:34.729657 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 410/807] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Oct 31 12:21:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 31 12:21:34.735465 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfe7020-f0f9-4018-b04a-d4ce2240963e None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:34.750129 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f30eae8-53b7-4e7e-9c65-41c4b62789e6 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:34.785060 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a4eeeb-baaa-4a90-9a4d-4c9766dada6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:34.785770 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a4eeeb-baaa-4a90-9a4d-4c9766dada6d None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:34.786123 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a4eeeb-baaa-4a90-9a4d-4c9766dada6d None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:34.786466 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a4eeeb-baaa-4a90-9a4d-4c9766dada6d None admin] PATH_INFO: `/v3/users/nova` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:34.794265 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a4eeeb-baaa-4a90-9a4d-4c9766dada6d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:34.795725 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a4eeeb-baaa-4a90-9a4d-4c9766dada6d None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:34.808018 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-e8a4eeeb-baaa-4a90-9a4d-4c9766dada6d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 31 12:21:34.808018 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:34.808018 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:21:34.808018 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:21:34.808018 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:21:34.808018 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:21:34.808018 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:21:34.808018 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:21:34.808018 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:21:34.808018 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:21:34.808018 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:21:34.808018 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:21:34.808018 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 31 12:21:34.808018 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 31 12:21:34.808018 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 31 12:21:34.808018 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 31 12:21:34.809110 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:21:34.809110 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:21:34.809110 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 31 12:21:34.809110 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 31 12:21:34.809110 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 31 12:21:34.809110 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 31 12:21:34.809110 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:21:34.809110 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:21:34.809110 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:21:34.809110 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:21:34.809110 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:21:34.809110 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:21:34.809110 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:21:34.809110 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:21:34.809110 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:21:34.809110 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:21:34.809110 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:21:34.809110 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:21:34.810246 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:21:34.810246 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:21:34.810246 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 31 12:21:34.810246 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 31 12:21:34.810246 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 31 12:21:34.810246 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 31 12:21:34.810246 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 31 12:21:34.810246 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 31 12:21:34.810246 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 31 12:21:34.810246 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:21:34.811313 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 398/808] 10.222.0.15 () {56 vars in 1093 bytes} [Mon Oct 31 12:21:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 31 12:21:34.814641 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a4eeeb-baaa-4a90-9a4d-4c9766dada6d None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:34.835554 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfe7020-f0f9-4018-b04a-d4ce2240963e None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:34.838025 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:34.870900 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4f177f-dc56-4a79-9bba-aba20ef23d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:34.871645 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4f177f-dc56-4a79-9bba-aba20ef23d33 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:34.871997 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4f177f-dc56-4a79-9bba-aba20ef23d33 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:34.872339 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4f177f-dc56-4a79-9bba-aba20ef23d33 None admin] PATH_INFO: `/v3/users/c0bf33b988ab44c9bf512fe1f6107d74` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:34.886058 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4f177f-dc56-4a79-9bba-aba20ef23d33 None admin] RBAC: Authorizing `identity:get_user(user_id=c0bf33b988ab44c9bf512fe1f6107d74)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:34.887546 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4f177f-dc56-4a79-9bba-aba20ef23d33 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:34.894121 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 411/809] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Oct 31 12:21:34 2022] GET /identity/v3/users/c0bf33b988ab44c9bf512fe1f6107d74 => generated 285 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 31 12:21:34.895106 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4f177f-dc56-4a79-9bba-aba20ef23d33 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:34.902937 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:34.916678 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a4eeeb-baaa-4a90-9a4d-4c9766dada6d None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:34.951600 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b13c4d-a320-4d46-9a16-6b1c06d10cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:34.952195 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b13c4d-a320-4d46-9a16-6b1c06d10cb5 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:34.952470 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b13c4d-a320-4d46-9a16-6b1c06d10cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:34.952737 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b13c4d-a320-4d46-9a16-6b1c06d10cb5 None admin] PATH_INFO: `/v3/users` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:34.953442 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b13c4d-a320-4d46-9a16-6b1c06d10cb5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:34.953740 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b13c4d-a320-4d46-9a16-6b1c06d10cb5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:34.955153 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b13c4d-a320-4d46-9a16-6b1c06d10cb5 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:34.973319 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 399/810] 10.222.0.15 () {56 vars in 1133 bytes} [Mon Oct 31 12:21:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 31 12:21:34.979133 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b13c4d-a320-4d46-9a16-6b1c06d10cb5 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:34.994742 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4f177f-dc56-4a79-9bba-aba20ef23d33 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:35.029854 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1852ae-057f-4924-b685-0e2ff55d20fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:35.030493 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1852ae-057f-4924-b685-0e2ff55d20fc None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:35.030766 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1852ae-057f-4924-b685-0e2ff55d20fc None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:35.031035 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1852ae-057f-4924-b685-0e2ff55d20fc None admin] PATH_INFO: `/v3/projects/c47da8e9550c40b4924e18f40718812f` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:35.032194 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1852ae-057f-4924-b685-0e2ff55d20fc None admin] RBAC: Authorizing `identity:get_project(project_id=c47da8e9550c40b4924e18f40718812f)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:35.033594 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1852ae-057f-4924-b685-0e2ff55d20fc None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:35.035225 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 412/811] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Oct 31 12:21:34 2022] GET /identity/v3/projects/c47da8e9550c40b4924e18f40718812f => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:21:35.040938 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1852ae-057f-4924-b685-0e2ff55d20fc None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:35.071728 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:35.078887 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b13c4d-a320-4d46-9a16-6b1c06d10cb5 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:35.112681 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3242d304-0e34-40cc-b5a2-3d5eba8b8509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:35.113303 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3242d304-0e34-40cc-b5a2-3d5eba8b8509 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:35.113570 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3242d304-0e34-40cc-b5a2-3d5eba8b8509 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:35.113834 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3242d304-0e34-40cc-b5a2-3d5eba8b8509 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:35.123647 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3242d304-0e34-40cc-b5a2-3d5eba8b8509 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:35.125069 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3242d304-0e34-40cc-b5a2-3d5eba8b8509 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:35.132252 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:35.140559 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-3242d304-0e34-40cc-b5a2-3d5eba8b8509 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:21:35.140559 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:35.140559 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:21:35.140559 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:21:35.140559 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:21:35.140559 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:21:35.140559 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:21:35.140559 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:21:35.140559 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:21:35.140559 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:21:35.140559 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:21:35.140559 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:21:35.140559 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 31 12:21:35.140559 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 31 12:21:35.140559 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 31 12:21:35.140559 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 31 12:21:35.141517 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:21:35.141517 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:21:35.141517 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:21:35.141517 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:21:35.141517 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:21:35.141517 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:21:35.141517 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:21:35.141517 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:21:35.141517 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:21:35.141517 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:21:35.141517 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:21:35.141517 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:21:35.141517 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:21:35.141517 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:21:35.141517 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:21:35.141517 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:21:35.141517 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 31 12:21:35.142411 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 31 12:21:35.142411 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:21:35.142411 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:21:35.142603 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1852ae-057f-4924-b685-0e2ff55d20fc None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:35.142685 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 400/812] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Oct 31 12:21:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 31 12:21:35.147560 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3242d304-0e34-40cc-b5a2-3d5eba8b8509 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:35.175742 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9104bea7-c0ae-4ee7-b12d-921d4d64e36e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:35.176382 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9104bea7-c0ae-4ee7-b12d-921d4d64e36e None admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:35.176602 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9104bea7-c0ae-4ee7-b12d-921d4d64e36e None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:35.176874 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9104bea7-c0ae-4ee7-b12d-921d4d64e36e None admin] PATH_INFO: `/v3/projects/c47da8e9550c40b4924e18f40718812f/users/c0bf33b988ab44c9bf512fe1f6107d74/roles/176c07b4fe3b4537bdfeb4be21b379c5` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:35.178759 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9104bea7-c0ae-4ee7-b12d-921d4d64e36e None admin] RBAC: Authorizing `identity:create_grant(project_id=c47da8e9550c40b4924e18f40718812f, user_id=c0bf33b988ab44c9bf512fe1f6107d74, role_id=176c07b4fe3b4537bdfeb4be21b379c5)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:35.180225 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9104bea7-c0ae-4ee7-b12d-921d4d64e36e None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:35.197756 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 413/813] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Oct 31 12:21:35 2022] PUT /identity/v3/projects/c47da8e9550c40b4924e18f40718812f/users/c0bf33b988ab44c9bf512fe1f6107d74/roles/176c07b4fe3b4537bdfeb4be21b379c5 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 31 12:21:35.256367 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3242d304-0e34-40cc-b5a2-3d5eba8b8509 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:35.293818 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d0810e-416f-43e0-b787-f16bad0edfc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:35.294432 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d0810e-416f-43e0-b787-f16bad0edfc8 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:35.294707 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d0810e-416f-43e0-b787-f16bad0edfc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:35.294969 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d0810e-416f-43e0-b787-f16bad0edfc8 None admin] PATH_INFO: `/v3/domains` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:35.295635 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d0810e-416f-43e0-b787-f16bad0edfc8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:35.295935 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d0810e-416f-43e0-b787-f16bad0edfc8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:35.297295 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d0810e-416f-43e0-b787-f16bad0edfc8 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:35.311002 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 401/814] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Oct 31 12:21:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 31 12:21:35.316905 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9104bea7-c0ae-4ee7-b12d-921d4d64e36e None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:35.412960 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:35.423645 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9104bea7-c0ae-4ee7-b12d-921d4d64e36e None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:35.456955 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0603a2-ee03-46f2-a44c-c1c94b73f221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:35.457509 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0603a2-ee03-46f2-a44c-c1c94b73f221 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:35.457785 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0603a2-ee03-46f2-a44c-c1c94b73f221 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:35.458052 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0603a2-ee03-46f2-a44c-c1c94b73f221 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:35.465067 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0603a2-ee03-46f2-a44c-c1c94b73f221 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:35.466523 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0603a2-ee03-46f2-a44c-c1c94b73f221 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:35.477326 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-bf0603a2-ee03-46f2-a44c-c1c94b73f221 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 31 12:21:35.477326 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:35.477326 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:21:35.477326 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:21:35.477326 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:21:35.477326 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:21:35.477326 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:21:35.477326 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:21:35.477326 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:21:35.477326 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:21:35.477326 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:21:35.477326 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:21:35.477326 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 31 12:21:35.477326 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 31 12:21:35.477326 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 31 12:21:35.478483 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 31 12:21:35.478483 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:21:35.478483 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:21:35.478483 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:21:35.478483 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:21:35.478483 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:21:35.478483 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:21:35.478483 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:21:35.478483 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 31 12:21:35.478483 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:21:35.478483 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:21:35.478483 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:21:35.478483 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:21:35.478483 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:21:35.478483 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:21:35.478483 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:21:35.478483 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:21:35.479571 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 31 12:21:35.479571 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 31 12:21:35.479571 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 31 12:21:35.479571 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 31 12:21:35.479571 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 31 12:21:35.479571 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 31 12:21:35.479571 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 31 12:21:35.479571 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:21:35.479571 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 414/815] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Oct 31 12:21:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:21:35.484271 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d0810e-416f-43e0-b787-f16bad0edfc8 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:35.542075 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:35.582690 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d0810e-416f-43e0-b787-f16bad0edfc8 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:35.616328 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f81a348-096c-4377-8a56-41935a21e1f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:35.616930 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f81a348-096c-4377-8a56-41935a21e1f4 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:35.617207 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f81a348-096c-4377-8a56-41935a21e1f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:35.617475 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f81a348-096c-4377-8a56-41935a21e1f4 None admin] PATH_INFO: `/v3/projects` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:35.618148 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f81a348-096c-4377-8a56-41935a21e1f4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:35.618759 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f81a348-096c-4377-8a56-41935a21e1f4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:35.619949 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f81a348-096c-4377-8a56-41935a21e1f4 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:35.631249 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 402/816] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Oct 31 12:21:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 31 12:21:35.638121 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0603a2-ee03-46f2-a44c-c1c94b73f221 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:35.736492 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0603a2-ee03-46f2-a44c-c1c94b73f221 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:35.770321 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f66a64b-bd1d-4119-b539-67be416c22f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:35.770971 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f66a64b-bd1d-4119-b539-67be416c22f1 None admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:35.771283 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f66a64b-bd1d-4119-b539-67be416c22f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:35.771557 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f66a64b-bd1d-4119-b539-67be416c22f1 None admin] PATH_INFO: `/v3/projects/2c32444cef584bce966b38203ce128b2/users/99889b1dab0b471d9c77887cffefc652/roles/92746653d6b94cf5a4e22d9efd5d4e4f` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:35.773656 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f66a64b-bd1d-4119-b539-67be416c22f1 None admin] RBAC: Authorizing `identity:create_grant(project_id=2c32444cef584bce966b38203ce128b2, user_id=99889b1dab0b471d9c77887cffefc652, role_id=92746653d6b94cf5a4e22d9efd5d4e4f)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:35.775127 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f66a64b-bd1d-4119-b539-67be416c22f1 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:35.787349 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 415/817] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Oct 31 12:21:35 2022] PUT /identity/v3/projects/2c32444cef584bce966b38203ce128b2/users/99889b1dab0b471d9c77887cffefc652/roles/92746653d6b94cf5a4e22d9efd5d4e4f => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:21:37.395342 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94549e5-8b95-4381-8e44-9017036e0959 None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:37.395858 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94549e5-8b95-4381-8e44-9017036e0959 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:37.395858 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94549e5-8b95-4381-8e44-9017036e0959 None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:37.397149 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 403/818] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:21:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:21:37.402661 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90040f24-3394-48b8-9330-12f6f7810d45 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:37.403077 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90040f24-3394-48b8-9330-12f6f7810d45 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:37.403305 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90040f24-3394-48b8-9330-12f6f7810d45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:37.421092 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-90040f24-3394-48b8-9330-12f6f7810d45 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:21:37.422247 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-90040f24-3394-48b8-9330-12f6f7810d45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:21:37.424954 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:37.488754 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 416/819] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:21:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 31 12:21:37.754932 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94549e5-8b95-4381-8e44-9017036e0959 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:37.861601 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94549e5-8b95-4381-8e44-9017036e0959 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:37.877823 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:37.895546 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed65764a-c2ed-45aa-80ae-46bf3af5d3fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:37.896171 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed65764a-c2ed-45aa-80ae-46bf3af5d3fe None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:37.896445 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed65764a-c2ed-45aa-80ae-46bf3af5d3fe None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:37.896709 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed65764a-c2ed-45aa-80ae-46bf3af5d3fe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:37.907624 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed65764a-c2ed-45aa-80ae-46bf3af5d3fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 31 12:21:37.910617 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad425af0-daf6-4993-a804-5c260a3e4a3a None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:37.910908 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad425af0-daf6-4993-a804-5c260a3e4a3a None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:37.911208 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad425af0-daf6-4993-a804-5c260a3e4a3a None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:37.912404 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-ed65764a-c2ed-45aa-80ae-46bf3af5d3fe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 31 12:21:37.912404 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:37.912404 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:21:37.912404 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:21:37.912404 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:21:37.912404 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:21:37.912404 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:21:37.912404 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:21:37.912404 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:21:37.912404 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:21:37.912404 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:21:37.912404 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:21:37.912404 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 31 12:21:37.912404 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 31 12:21:37.912404 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 31 12:21:37.912404 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 31 12:21:37.913309 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 31 12:21:37.913309 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 31 12:21:37.913309 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 31 12:21:37.913309 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 31 12:21:37.913309 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:21:37.913309 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:21:37.913309 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:21:37.913309 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:21:37.913309 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:21:37.913309 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:21:37.913309 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:21:37.913309 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:21:37.913309 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:21:37.913309 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:21:37.913309 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:21:37.913309 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:21:37.914157 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:21:37.914157 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:21:37.914157 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:21:37.914157 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:21:37.914157 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 31 12:21:37.914157 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 31 12:21:37.914157 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 31 12:21:37.914157 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 31 12:21:37.914157 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 31 12:21:37.914157 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 31 12:21:37.914157 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 31 12:21:37.914157 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:21:37.914800 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 417/820] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:21:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:21:37.914876 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 404/821] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Oct 31 12:21:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:21:37.917985 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbe3752-9ace-41ea-9f3b-1cbb1be992c0 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:37.918336 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbe3752-9ace-41ea-9f3b-1cbb1be992c0 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:37.918610 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbe3752-9ace-41ea-9f3b-1cbb1be992c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:37.919218 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed65764a-c2ed-45aa-80ae-46bf3af5d3fe None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:37.935963 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-cdbe3752-9ace-41ea-9f3b-1cbb1be992c0 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:21:37.937132 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-cdbe3752-9ace-41ea-9f3b-1cbb1be992c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:21:38.003509 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 418/822] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:21:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 31 12:21:38.016970 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed65764a-c2ed-45aa-80ae-46bf3af5d3fe None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:38.053076 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d877ad2-1ea0-4e75-b752-dfdda20d2408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:38.053670 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d877ad2-1ea0-4e75-b752-dfdda20d2408 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:38.053945 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d877ad2-1ea0-4e75-b752-dfdda20d2408 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:38.054210 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d877ad2-1ea0-4e75-b752-dfdda20d2408 None admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:38.054852 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d877ad2-1ea0-4e75-b752-dfdda20d2408 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:38.055146 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d877ad2-1ea0-4e75-b752-dfdda20d2408 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:38.056857 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d877ad2-1ea0-4e75-b752-dfdda20d2408 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:38.065077 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 405/823] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Oct 31 12:21:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 31 12:21:38.070908 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbe3752-9ace-41ea-9f3b-1cbb1be992c0 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:38.137695 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:38.168301 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbe3752-9ace-41ea-9f3b-1cbb1be992c0 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:38.201372 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ef628a-0893-432c-91d3-6e17dd7e035f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:38.202003 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ef628a-0893-432c-91d3-6e17dd7e035f None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:38.202284 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ef628a-0893-432c-91d3-6e17dd7e035f None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:38.202558 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ef628a-0893-432c-91d3-6e17dd7e035f None admin] PATH_INFO: `/v3/users/c0bf33b988ab44c9bf512fe1f6107d74` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:38.203702 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ef628a-0893-432c-91d3-6e17dd7e035f None admin] RBAC: Authorizing `identity:get_user(user_id=c0bf33b988ab44c9bf512fe1f6107d74)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:38.205115 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ef628a-0893-432c-91d3-6e17dd7e035f None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:38.206677 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 419/824] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Oct 31 12:21:38 2022] GET /identity/v3/users/c0bf33b988ab44c9bf512fe1f6107d74 => generated 285 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:21:38.212898 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d877ad2-1ea0-4e75-b752-dfdda20d2408 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:38.253454 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:38.272266 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ef628a-0893-432c-91d3-6e17dd7e035f None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:38.312044 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d877ad2-1ea0-4e75-b752-dfdda20d2408 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:38.345523 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccd24e0-11d5-4513-8a88-2eb3069cd8b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:38.346121 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccd24e0-11d5-4513-8a88-2eb3069cd8b0 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:38.346393 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccd24e0-11d5-4513-8a88-2eb3069cd8b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:38.346658 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccd24e0-11d5-4513-8a88-2eb3069cd8b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:38.370687 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ef628a-0893-432c-91d3-6e17dd7e035f None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:38.405832 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2530d3-4f19-4873-998f-04cca85da32c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:38.406455 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2530d3-4f19-4873-998f-04cca85da32c None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:38.406722 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2530d3-4f19-4873-998f-04cca85da32c None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:38.406992 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2530d3-4f19-4873-998f-04cca85da32c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:38.413238 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccd24e0-11d5-4513-8a88-2eb3069cd8b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:38.413785 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:38.414711 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccd24e0-11d5-4513-8a88-2eb3069cd8b0 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:38.418456 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2530d3-4f19-4873-998f-04cca85da32c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 31 12:21:38.423501 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-ec2530d3-4f19-4873-998f-04cca85da32c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 31 12:21:38.423501 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:38.423501 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:21:38.423501 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:21:38.423501 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:21:38.423501 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:21:38.423501 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:21:38.423501 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:21:38.423501 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:21:38.423501 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:21:38.423501 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:21:38.423501 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:21:38.423501 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 31 12:21:38.423501 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 31 12:21:38.423501 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 31 12:21:38.423501 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 31 12:21:38.424585 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 31 12:21:38.424585 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 31 12:21:38.424585 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 31 12:21:38.424585 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 31 12:21:38.424585 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:21:38.424585 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:21:38.424585 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:21:38.424585 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:21:38.424585 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:21:38.424585 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:21:38.424585 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:21:38.424585 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 31 12:21:38.424585 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:21:38.424585 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:21:38.424585 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:21:38.424585 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:21:38.425453 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:21:38.425453 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:21:38.425453 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:21:38.425453 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:21:38.425453 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 31 12:21:38.425453 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 31 12:21:38.425453 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 31 12:21:38.425453 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 31 12:21:38.425453 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 31 12:21:38.425453 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 31 12:21:38.425453 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 31 12:21:38.425453 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:21:38.425453 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 420/825] 10.222.0.15 () {56 vars in 1120 bytes} [Mon Oct 31 12:21:38 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:21:38.430536 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2530d3-4f19-4873-998f-04cca85da32c None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:38.457416 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:38.520416 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 406/826] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Oct 31 12:21:38 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 31 12:21:38.526721 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccd24e0-11d5-4513-8a88-2eb3069cd8b0 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:38.624987 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccd24e0-11d5-4513-8a88-2eb3069cd8b0 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:38.644832 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2530d3-4f19-4873-998f-04cca85da32c None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:38.663154 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6a335f-5f86-4921-b806-28a566b55361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:38.663798 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6a335f-5f86-4921-b806-28a566b55361 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:38.664077 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6a335f-5f86-4921-b806-28a566b55361 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:38.664351 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6a335f-5f86-4921-b806-28a566b55361 None admin] PATH_INFO: `/v3/projects/c47da8e9550c40b4924e18f40718812f` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:38.666567 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6a335f-5f86-4921-b806-28a566b55361 None admin] RBAC: Authorizing `identity:get_project(project_id=c47da8e9550c40b4924e18f40718812f)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:38.668020 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6a335f-5f86-4921-b806-28a566b55361 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:38.669642 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 407/827] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Oct 31 12:21:38 2022] GET /identity/v3/projects/c47da8e9550c40b4924e18f40718812f => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 31 12:21:38.675775 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6a335f-5f86-4921-b806-28a566b55361 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:38.680012 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dc8196-fa58-4cc9-a565-9b20147252e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:38.681384 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dc8196-fa58-4cc9-a565-9b20147252e9 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:38.681668 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dc8196-fa58-4cc9-a565-9b20147252e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:38.682082 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dc8196-fa58-4cc9-a565-9b20147252e9 None admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:38.682789 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dc8196-fa58-4cc9-a565-9b20147252e9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:38.683086 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dc8196-fa58-4cc9-a565-9b20147252e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:38.684462 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dc8196-fa58-4cc9-a565-9b20147252e9 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:38.687605 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:38.692547 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 421/828] 10.222.0.15 () {56 vars in 1115 bytes} [Mon Oct 31 12:21:38 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 31 12:21:38.698320 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dc8196-fa58-4cc9-a565-9b20147252e9 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:38.757878 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:38.780392 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6a335f-5f86-4921-b806-28a566b55361 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:38.797574 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dc8196-fa58-4cc9-a565-9b20147252e9 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:38.813674 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d0aaeb-247e-4b46-804d-e7d2ee5ca805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:38.814273 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d0aaeb-247e-4b46-804d-e7d2ee5ca805 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:38.814549 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d0aaeb-247e-4b46-804d-e7d2ee5ca805 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:38.814816 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d0aaeb-247e-4b46-804d-e7d2ee5ca805 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:38.815561 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d0aaeb-247e-4b46-804d-e7d2ee5ca805 None admin] RBAC: Adding query filter params (role.id=176c07b4fe3b4537bdfeb4be21b379c5, scope.project.id=c47da8e9550c40b4924e18f40718812f, user.id=c0bf33b988ab44c9bf512fe1f6107d74) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:38.815862 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d0aaeb-247e-4b46-804d-e7d2ee5ca805 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:38.817364 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d0aaeb-247e-4b46-804d-e7d2ee5ca805 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:38.823060 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 408/829] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Oct 31 12:21:38 2022] GET /identity/v3/role_assignments?user.id=c0bf33b988ab44c9bf512fe1f6107d74&scope.project.id=c47da8e9550c40b4924e18f40718812f&role.id=176c07b4fe3b4537bdfeb4be21b379c5 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 31 12:21:38.830186 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6583748c-4ea2-48c4-bc21-cf8e33151e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:38.830842 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6583748c-4ea2-48c4-bc21-cf8e33151e57 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:38.831114 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6583748c-4ea2-48c4-bc21-cf8e33151e57 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:38.831410 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6583748c-4ea2-48c4-bc21-cf8e33151e57 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:38.843205 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6583748c-4ea2-48c4-bc21-cf8e33151e57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:38.844671 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6583748c-4ea2-48c4-bc21-cf8e33151e57 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:38.861143 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-6583748c-4ea2-48c4-bc21-cf8e33151e57 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:21:38.861143 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:38.861143 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:21:38.861143 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:21:38.861143 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:21:38.861143 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:21:38.861143 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:21:38.861143 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:21:38.861143 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:21:38.861143 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:21:38.861143 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:21:38.861143 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:21:38.861143 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 31 12:21:38.861143 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 31 12:21:38.861143 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 31 12:21:38.861143 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 31 12:21:38.862339 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:21:38.862339 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:21:38.862339 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:21:38.862339 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:21:38.862339 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:21:38.862339 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:21:38.862339 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:21:38.862339 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 31 12:21:38.862339 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:21:38.862339 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:21:38.862339 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:21:38.862339 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:21:38.862339 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:21:38.862339 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:21:38.862339 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:21:38.862339 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:21:38.862339 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 31 12:21:38.863271 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 31 12:21:38.863271 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:21:38.863271 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:21:38.863271 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 422/830] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Oct 31 12:21:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 31 12:21:38.868307 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d0aaeb-247e-4b46-804d-e7d2ee5ca805 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:38.943291 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:38.975807 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d0aaeb-247e-4b46-804d-e7d2ee5ca805 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:39.011425 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aba49a-5571-43a0-a186-5e44d7545b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:39.012028 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aba49a-5571-43a0-a186-5e44d7545b5b None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:39.012306 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aba49a-5571-43a0-a186-5e44d7545b5b None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:39.012573 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aba49a-5571-43a0-a186-5e44d7545b5b None admin] PATH_INFO: `/v3/domains` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:39.013199 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64aba49a-5571-43a0-a186-5e44d7545b5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:39.013503 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64aba49a-5571-43a0-a186-5e44d7545b5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:39.014836 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64aba49a-5571-43a0-a186-5e44d7545b5b None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:39.029201 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 409/831] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Oct 31 12:21:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:21:39.035035 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6583748c-4ea2-48c4-bc21-cf8e33151e57 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:39.132935 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6583748c-4ea2-48c4-bc21-cf8e33151e57 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:39.146690 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:39.168310 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9c030e-7dba-4af6-8227-b3ff7912f282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:39.168934 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9c030e-7dba-4af6-8227-b3ff7912f282 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:39.169209 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9c030e-7dba-4af6-8227-b3ff7912f282 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:39.169476 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9c030e-7dba-4af6-8227-b3ff7912f282 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:39.176373 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9c030e-7dba-4af6-8227-b3ff7912f282 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:39.177781 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9c030e-7dba-4af6-8227-b3ff7912f282 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:39.189223 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-ff9c030e-7dba-4af6-8227-b3ff7912f282 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 31 12:21:39.189223 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:39.189223 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:21:39.189223 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:21:39.189223 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:21:39.189223 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:21:39.189223 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:21:39.189223 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:21:39.189223 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:21:39.189223 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:21:39.189223 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:21:39.189223 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:21:39.189223 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 31 12:21:39.189223 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 31 12:21:39.189223 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 31 12:21:39.189223 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 31 12:21:39.190355 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:21:39.190355 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:21:39.190355 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 31 12:21:39.190355 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 31 12:21:39.190355 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 31 12:21:39.190355 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 31 12:21:39.190355 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:21:39.190355 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:21:39.190355 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:21:39.190355 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:21:39.190355 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:21:39.190355 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 31 12:21:39.190355 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:21:39.190355 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:21:39.190355 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:21:39.190355 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:21:39.190355 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:21:39.190355 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:21:39.191549 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:21:39.191549 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:21:39.191549 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 31 12:21:39.191549 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 31 12:21:39.191549 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 31 12:21:39.191549 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 31 12:21:39.191549 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 31 12:21:39.191549 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 31 12:21:39.191549 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 31 12:21:39.191549 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:21:39.192348 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 423/832] 10.222.0.15 () {56 vars in 1093 bytes} [Mon Oct 31 12:21:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:21:39.196826 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aba49a-5571-43a0-a186-5e44d7545b5b None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:39.294525 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aba49a-5571-43a0-a186-5e44d7545b5b None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:39.323735 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:39.329329 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fb4326-93ee-4484-8d70-b27b3e6f9452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:39.329936 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fb4326-93ee-4484-8d70-b27b3e6f9452 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:39.330227 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fb4326-93ee-4484-8d70-b27b3e6f9452 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:39.330491 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fb4326-93ee-4484-8d70-b27b3e6f9452 None admin] PATH_INFO: `/v3/users` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:39.331231 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fb4326-93ee-4484-8d70-b27b3e6f9452 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:39.331532 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fb4326-93ee-4484-8d70-b27b3e6f9452 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:39.332931 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fb4326-93ee-4484-8d70-b27b3e6f9452 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:39.349928 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 410/833] 10.222.0.15 () {56 vars in 1133 bytes} [Mon Oct 31 12:21:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:21:39.356113 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9c030e-7dba-4af6-8227-b3ff7912f282 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:39.456062 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9c030e-7dba-4af6-8227-b3ff7912f282 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:39.489397 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b988f6fe-de9c-456b-9776-1ea9263733c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:39.489706 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b988f6fe-de9c-456b-9776-1ea9263733c6 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:39.489982 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b988f6fe-de9c-456b-9776-1ea9263733c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:39.490249 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b988f6fe-de9c-456b-9776-1ea9263733c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:39.530263 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:39.557636 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b988f6fe-de9c-456b-9776-1ea9263733c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:39.559018 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b988f6fe-de9c-456b-9776-1ea9263733c6 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:39.654399 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 424/834] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Oct 31 12:21:39 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 31 12:21:39.660192 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fb4326-93ee-4484-8d70-b27b3e6f9452 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:39.760550 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fb4326-93ee-4484-8d70-b27b3e6f9452 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:39.793917 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52f1159-6150-4a09-824b-44e64a299619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:39.794536 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52f1159-6150-4a09-824b-44e64a299619 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:39.794811 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52f1159-6150-4a09-824b-44e64a299619 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:39.795078 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52f1159-6150-4a09-824b-44e64a299619 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:39.804611 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52f1159-6150-4a09-824b-44e64a299619 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:39.806566 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52f1159-6150-4a09-824b-44e64a299619 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:39.820720 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-f52f1159-6150-4a09-824b-44e64a299619 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:21:39.820720 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:39.820720 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:21:39.820720 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:21:39.820720 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:21:39.820720 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:21:39.820720 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:21:39.820720 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:21:39.820720 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:21:39.820720 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:21:39.820720 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:21:39.820720 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:21:39.820720 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 31 12:21:39.820720 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 31 12:21:39.820720 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 31 12:21:39.820720 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 31 12:21:39.821795 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:21:39.821795 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:21:39.821795 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:21:39.821795 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:21:39.821795 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:21:39.821795 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:21:39.821795 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:21:39.821795 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:21:39.821795 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:21:39.821795 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:21:39.821795 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:21:39.821795 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:21:39.821795 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:21:39.821795 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:21:39.821795 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:21:39.821795 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:21:39.821795 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 31 12:21:39.822776 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 31 12:21:39.822776 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:21:39.822776 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:21:39.822776 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 411/835] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Oct 31 12:21:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:21:39.827936 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b988f6fe-de9c-456b-9776-1ea9263733c6 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:39.894653 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:39.923274 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b988f6fe-de9c-456b-9776-1ea9263733c6 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:39.955915 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c327367a-cb1e-42a7-b874-0095f897c768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:39.956514 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c327367a-cb1e-42a7-b874-0095f897c768 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:39.956788 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c327367a-cb1e-42a7-b874-0095f897c768 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:39.957051 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c327367a-cb1e-42a7-b874-0095f897c768 None admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:39.957678 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c327367a-cb1e-42a7-b874-0095f897c768 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:39.957980 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c327367a-cb1e-42a7-b874-0095f897c768 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:39.959381 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c327367a-cb1e-42a7-b874-0095f897c768 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:39.970758 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 425/836] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Oct 31 12:21:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:21:39.976453 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52f1159-6150-4a09-824b-44e64a299619 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:40.019914 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:40.080383 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52f1159-6150-4a09-824b-44e64a299619 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:40.113337 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65419b74-8d25-4ae2-be10-7c0c57a78041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:40.113950 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65419b74-8d25-4ae2-be10-7c0c57a78041 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:40.114223 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65419b74-8d25-4ae2-be10-7c0c57a78041 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:40.114487 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65419b74-8d25-4ae2-be10-7c0c57a78041 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:40.120925 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65419b74-8d25-4ae2-be10-7c0c57a78041 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:40.122352 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65419b74-8d25-4ae2-be10-7c0c57a78041 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:40.133192 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-65419b74-8d25-4ae2-be10-7c0c57a78041 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 31 12:21:40.133192 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:40.133192 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:21:40.133192 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:21:40.133192 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:21:40.133192 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:21:40.133192 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:21:40.133192 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:21:40.133192 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:21:40.133192 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:21:40.133192 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:21:40.133192 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:21:40.133192 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 31 12:21:40.133192 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 31 12:21:40.133192 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 31 12:21:40.134795 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 31 12:21:40.134795 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:21:40.134795 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:21:40.134795 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:21:40.134795 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:21:40.134795 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:21:40.134795 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:21:40.134795 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:21:40.134795 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:21:40.134795 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:21:40.134795 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:21:40.134795 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:21:40.134795 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:21:40.134795 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:21:40.134795 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:21:40.134795 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:21:40.134795 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:21:40.135863 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 31 12:21:40.135863 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 31 12:21:40.135863 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 31 12:21:40.135863 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 31 12:21:40.135863 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 31 12:21:40.135863 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 31 12:21:40.135863 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 31 12:21:40.135863 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:21:40.135863 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 412/837] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Oct 31 12:21:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:21:40.140696 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c327367a-cb1e-42a7-b874-0095f897c768 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:40.236180 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c327367a-cb1e-42a7-b874-0095f897c768 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:40.264419 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:40.270047 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479636b5-bfef-4079-a17b-d1c6fb85124c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:40.270645 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479636b5-bfef-4079-a17b-d1c6fb85124c None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:40.270920 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479636b5-bfef-4079-a17b-d1c6fb85124c None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:40.271215 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479636b5-bfef-4079-a17b-d1c6fb85124c None admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:40.271889 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479636b5-bfef-4079-a17b-d1c6fb85124c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:40.272188 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479636b5-bfef-4079-a17b-d1c6fb85124c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:40.273582 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479636b5-bfef-4079-a17b-d1c6fb85124c None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:40.285140 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 426/838] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Oct 31 12:21:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:21:40.292055 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65419b74-8d25-4ae2-be10-7c0c57a78041 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:40.389236 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65419b74-8d25-4ae2-be10-7c0c57a78041 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:40.390661 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:40.423440 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd7f30a-5b32-4c24-ad5c-284cf17425b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:40.424035 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd7f30a-5b32-4c24-ad5c-284cf17425b4 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:40.424309 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd7f30a-5b32-4c24-ad5c-284cf17425b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:40.424576 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd7f30a-5b32-4c24-ad5c-284cf17425b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:40.425276 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd7f30a-5b32-4c24-ad5c-284cf17425b4 None admin] RBAC: Adding query filter params (role.id=92746653d6b94cf5a4e22d9efd5d4e4f, scope.project.id=2c32444cef584bce966b38203ce128b2, user.id=99889b1dab0b471d9c77887cffefc652) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:40.425577 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd7f30a-5b32-4c24-ad5c-284cf17425b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:40.426976 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd7f30a-5b32-4c24-ad5c-284cf17425b4 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:40.433150 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 413/839] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Oct 31 12:21:40 2022] GET /identity/v3/role_assignments?user.id=99889b1dab0b471d9c77887cffefc652&scope.project.id=2c32444cef584bce966b38203ce128b2&role.id=92746653d6b94cf5a4e22d9efd5d4e4f => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:21:41.048718 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57ab05a-a799-496d-a655-e78174edfaf8 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:41.049216 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57ab05a-a799-496d-a655-e78174edfaf8 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:41.049300 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57ab05a-a799-496d-a655-e78174edfaf8 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:41.050585 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 427/840] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:21:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:21:41.056158 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57d143d-6f15-4fe6-8428-77d1034dbc45 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:41.056619 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57d143d-6f15-4fe6-8428-77d1034dbc45 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:41.056717 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57d143d-6f15-4fe6-8428-77d1034dbc45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:41.075184 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-c57d143d-6f15-4fe6-8428-77d1034dbc45 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:21:41.076383 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-c57d143d-6f15-4fe6-8428-77d1034dbc45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:21:41.144335 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 414/841] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:21:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 31 12:21:41.420924 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57ab05a-a799-496d-a655-e78174edfaf8 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:41.517758 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57ab05a-a799-496d-a655-e78174edfaf8 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:41.549836 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c82fcd7-0f00-4f13-8a0d-b9b06087829b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:41.550436 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c82fcd7-0f00-4f13-8a0d-b9b06087829b None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:41.550707 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c82fcd7-0f00-4f13-8a0d-b9b06087829b None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:41.550971 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c82fcd7-0f00-4f13-8a0d-b9b06087829b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:41.552077 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c82fcd7-0f00-4f13-8a0d-b9b06087829b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:41.553477 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c82fcd7-0f00-4f13-8a0d-b9b06087829b None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:41.554983 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 428/842] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Oct 31 12:21:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:21:41.560821 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57d143d-6f15-4fe6-8428-77d1034dbc45 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:41.650819 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:41.656919 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57d143d-6f15-4fe6-8428-77d1034dbc45 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:41.689171 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f33890-972f-48df-8b2d-e176f405c08f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:41.689701 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f33890-972f-48df-8b2d-e176f405c08f None admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:41.689975 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f33890-972f-48df-8b2d-e176f405c08f None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:41.690239 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f33890-972f-48df-8b2d-e176f405c08f None admin] PATH_INFO: `/v3/users` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:41.690864 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f33890-972f-48df-8b2d-e176f405c08f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:41.692296 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f33890-972f-48df-8b2d-e176f405c08f None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:41.713225 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 415/843] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Oct 31 12:21:41 2022] POST /identity/v3/users => generated 286 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:21:43.914222 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edbdae7-3cfc-49db-ac26-a7c1992c4f25 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:43.914751 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edbdae7-3cfc-49db-ac26-a7c1992c4f25 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:43.914751 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edbdae7-3cfc-49db-ac26-a7c1992c4f25 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:43.916732 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 429/844] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:21:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:21:43.921956 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d7b2af-bd36-44ea-beb8-eb49e66280f1 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:43.922353 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d7b2af-bd36-44ea-beb8-eb49e66280f1 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:43.922499 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d7b2af-bd36-44ea-beb8-eb49e66280f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:43.940763 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-f7d7b2af-bd36-44ea-beb8-eb49e66280f1 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:21:43.941947 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-f7d7b2af-bd36-44ea-beb8-eb49e66280f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:21:44.008293 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 416/845] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:21:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 31 12:21:44.282320 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edbdae7-3cfc-49db-ac26-a7c1992c4f25 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:44.340928 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:44.380283 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edbdae7-3cfc-49db-ac26-a7c1992c4f25 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:44.411826 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039c61e2-b71f-420a-82fe-68b091a47a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:44.412434 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039c61e2-b71f-420a-82fe-68b091a47a4a None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:44.412705 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039c61e2-b71f-420a-82fe-68b091a47a4a None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:44.412968 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039c61e2-b71f-420a-82fe-68b091a47a4a None admin] PATH_INFO: `/v3/roles/65728d8bb0e546788426da322c34baba` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:44.414310 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039c61e2-b71f-420a-82fe-68b091a47a4a None admin] RBAC: Authorizing `identity:get_role(role_id=65728d8bb0e546788426da322c34baba)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:44.416209 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039c61e2-b71f-420a-82fe-68b091a47a4a None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:44.417477 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 430/846] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Oct 31 12:21:44 2022] GET /identity/v3/roles/65728d8bb0e546788426da322c34baba => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:21:44.423259 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d7b2af-bd36-44ea-beb8-eb49e66280f1 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:44.502260 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:44.521728 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d7b2af-bd36-44ea-beb8-eb49e66280f1 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:44.554362 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51777fd2-2526-4e71-a1f1-142c9ce51cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:44.554923 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51777fd2-2526-4e71-a1f1-142c9ce51cd4 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:44.555223 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51777fd2-2526-4e71-a1f1-142c9ce51cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:44.555496 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51777fd2-2526-4e71-a1f1-142c9ce51cd4 None admin] PATH_INFO: `/v3/users/e674f30b9a594dc9b7cc1d9874291188` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:44.575951 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51777fd2-2526-4e71-a1f1-142c9ce51cd4 None admin] RBAC: Authorizing `identity:get_user(user_id=e674f30b9a594dc9b7cc1d9874291188)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:44.577364 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51777fd2-2526-4e71-a1f1-142c9ce51cd4 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:44.578931 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 417/847] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Oct 31 12:21:44 2022] GET /identity/v3/users/e674f30b9a594dc9b7cc1d9874291188 => generated 286 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:21:44.584597 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039c61e2-b71f-420a-82fe-68b091a47a4a None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:44.683613 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039c61e2-b71f-420a-82fe-68b091a47a4a None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:44.711013 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:44.719557 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6d090a-b71b-47d7-b024-363f102130c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:44.720173 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6d090a-b71b-47d7-b024-363f102130c1 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:44.720445 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6d090a-b71b-47d7-b024-363f102130c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:44.720710 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6d090a-b71b-47d7-b024-363f102130c1 None admin] PATH_INFO: `/v3/projects/c47da8e9550c40b4924e18f40718812f` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:44.721793 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6d090a-b71b-47d7-b024-363f102130c1 None admin] RBAC: Authorizing `identity:get_project(project_id=c47da8e9550c40b4924e18f40718812f)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:44.723207 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6d090a-b71b-47d7-b024-363f102130c1 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:44.724791 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 431/848] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Oct 31 12:21:44 2022] GET /identity/v3/projects/c47da8e9550c40b4924e18f40718812f => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:21:44.730821 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51777fd2-2526-4e71-a1f1-142c9ce51cd4 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:44.826163 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51777fd2-2526-4e71-a1f1-142c9ce51cd4 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:44.858212 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1cff37-8cd4-46c8-9252-8a53ee2f6b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:44.858839 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1cff37-8cd4-46c8-9252-8a53ee2f6b44 None admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:44.859114 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1cff37-8cd4-46c8-9252-8a53ee2f6b44 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:44.859411 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1cff37-8cd4-46c8-9252-8a53ee2f6b44 None admin] PATH_INFO: `/v3/projects/c47da8e9550c40b4924e18f40718812f/users/e674f30b9a594dc9b7cc1d9874291188/roles/65728d8bb0e546788426da322c34baba` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:44.861475 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1cff37-8cd4-46c8-9252-8a53ee2f6b44 None admin] RBAC: Authorizing `identity:create_grant(project_id=c47da8e9550c40b4924e18f40718812f, user_id=e674f30b9a594dc9b7cc1d9874291188, role_id=65728d8bb0e546788426da322c34baba)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:44.862871 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1cff37-8cd4-46c8-9252-8a53ee2f6b44 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:44.872005 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:44.876709 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 418/849] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Oct 31 12:21:44 2022] PUT /identity/v3/projects/c47da8e9550c40b4924e18f40718812f/users/e674f30b9a594dc9b7cc1d9874291188/roles/65728d8bb0e546788426da322c34baba => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:21:47.098154 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c4c17b-085e-4b7c-b337-6c83f31fec68 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:47.098629 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c4c17b-085e-4b7c-b337-6c83f31fec68 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:47.098714 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c4c17b-085e-4b7c-b337-6c83f31fec68 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:47.100040 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 432/850] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:21:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:21:47.105897 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108be87a-3738-4285-9984-7c55049c0871 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:47.106314 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108be87a-3738-4285-9984-7c55049c0871 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:47.106419 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108be87a-3738-4285-9984-7c55049c0871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:47.124097 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-108be87a-3738-4285-9984-7c55049c0871 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:21:47.125257 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-108be87a-3738-4285-9984-7c55049c0871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:21:47.188737 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 419/851] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:21:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 31 12:21:47.458333 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c4c17b-085e-4b7c-b337-6c83f31fec68 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:47.555073 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c4c17b-085e-4b7c-b337-6c83f31fec68 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:47.588420 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9094f063-992f-421e-8716-8cbb38e7d755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:47.589024 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9094f063-992f-421e-8716-8cbb38e7d755 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:47.589297 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9094f063-992f-421e-8716-8cbb38e7d755 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:47.589562 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9094f063-992f-421e-8716-8cbb38e7d755 None admin] PATH_INFO: `/v3/roles/65728d8bb0e546788426da322c34baba` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:47.591065 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9094f063-992f-421e-8716-8cbb38e7d755 None admin] RBAC: Authorizing `identity:get_role(role_id=65728d8bb0e546788426da322c34baba)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:47.592457 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9094f063-992f-421e-8716-8cbb38e7d755 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:47.593769 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 433/852] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Oct 31 12:21:47 2022] GET /identity/v3/roles/65728d8bb0e546788426da322c34baba => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:21:47.599452 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108be87a-3738-4285-9984-7c55049c0871 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:47.694160 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108be87a-3738-4285-9984-7c55049c0871 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:47.722510 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:47.728485 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8773e56-8dfa-46d8-bb8f-a090ef958b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:47.729094 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8773e56-8dfa-46d8-bb8f-a090ef958b55 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:47.729367 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8773e56-8dfa-46d8-bb8f-a090ef958b55 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:47.729633 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8773e56-8dfa-46d8-bb8f-a090ef958b55 None admin] PATH_INFO: `/v3/users/e674f30b9a594dc9b7cc1d9874291188` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:47.730719 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8773e56-8dfa-46d8-bb8f-a090ef958b55 None admin] RBAC: Authorizing `identity:get_user(user_id=e674f30b9a594dc9b7cc1d9874291188)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:47.732146 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8773e56-8dfa-46d8-bb8f-a090ef958b55 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:47.733687 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 420/853] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Oct 31 12:21:47 2022] GET /identity/v3/users/e674f30b9a594dc9b7cc1d9874291188 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:21:47.739493 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9094f063-992f-421e-8716-8cbb38e7d755 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:47.799901 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:47.837327 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9094f063-992f-421e-8716-8cbb38e7d755 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:47.869458 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489d0411-e227-4791-a285-9db7d90c1a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:47.870038 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489d0411-e227-4791-a285-9db7d90c1a96 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:47.870312 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489d0411-e227-4791-a285-9db7d90c1a96 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:47.870577 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489d0411-e227-4791-a285-9db7d90c1a96 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:47.934726 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489d0411-e227-4791-a285-9db7d90c1a96 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:47.936162 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489d0411-e227-4791-a285-9db7d90c1a96 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:47.993806 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:48.032702 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 434/854] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Oct 31 12:21:47 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 31 12:21:48.038501 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8773e56-8dfa-46d8-bb8f-a090ef958b55 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:48.134999 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8773e56-8dfa-46d8-bb8f-a090ef958b55 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:48.168459 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcffbf3a-0fff-4331-b9c8-664246a6c1a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:48.169107 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcffbf3a-0fff-4331-b9c8-664246a6c1a1 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:48.169387 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcffbf3a-0fff-4331-b9c8-664246a6c1a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:48.169659 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcffbf3a-0fff-4331-b9c8-664246a6c1a1 None admin] PATH_INFO: `/v3/projects/c47da8e9550c40b4924e18f40718812f` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:48.170753 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcffbf3a-0fff-4331-b9c8-664246a6c1a1 None admin] RBAC: Authorizing `identity:get_project(project_id=c47da8e9550c40b4924e18f40718812f)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:48.172201 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcffbf3a-0fff-4331-b9c8-664246a6c1a1 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:48.173810 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 421/855] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Oct 31 12:21:48 2022] GET /identity/v3/projects/c47da8e9550c40b4924e18f40718812f => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:21:48.179912 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489d0411-e227-4791-a285-9db7d90c1a96 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:48.276050 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489d0411-e227-4791-a285-9db7d90c1a96 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:48.308693 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39b102f-df09-4470-a095-e6cc85c9c143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:48.309276 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39b102f-df09-4470-a095-e6cc85c9c143 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:48.309550 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39b102f-df09-4470-a095-e6cc85c9c143 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:48.309817 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39b102f-df09-4470-a095-e6cc85c9c143 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:48.310543 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39b102f-df09-4470-a095-e6cc85c9c143 None admin] RBAC: Adding query filter params (role.id=65728d8bb0e546788426da322c34baba, scope.project.id=c47da8e9550c40b4924e18f40718812f, user.id=e674f30b9a594dc9b7cc1d9874291188) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:48.310826 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39b102f-df09-4470-a095-e6cc85c9c143 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:48.312239 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39b102f-df09-4470-a095-e6cc85c9c143 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:48.317726 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 435/856] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Oct 31 12:21:48 2022] GET /identity/v3/role_assignments?user.id=e674f30b9a594dc9b7cc1d9874291188&scope.project.id=c47da8e9550c40b4924e18f40718812f&role.id=65728d8bb0e546788426da322c34baba => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:21:50.458495 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3d72a0-396c-4142-9ac4-de9b806ce470 None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:50.458994 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3d72a0-396c-4142-9ac4-de9b806ce470 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:50.458994 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3d72a0-396c-4142-9ac4-de9b806ce470 None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:50.460321 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 422/857] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:21:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:21:50.465693 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9dd3d1-a7e9-4e9a-b664-1cffd8fb2206 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:50.466070 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9dd3d1-a7e9-4e9a-b664-1cffd8fb2206 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:50.466236 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9dd3d1-a7e9-4e9a-b664-1cffd8fb2206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:50.484510 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-3e9dd3d1-a7e9-4e9a-b664-1cffd8fb2206 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:21:50.485674 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-3e9dd3d1-a7e9-4e9a-b664-1cffd8fb2206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:21:50.508217 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:50.551873 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 436/858] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:21:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 31 12:21:50.822361 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3d72a0-396c-4142-9ac4-de9b806ce470 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:50.888529 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:50.924087 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3d72a0-396c-4142-9ac4-de9b806ce470 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:50.956899 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a511056-1d71-4efb-9ea0-2e96a3f7cf5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:50.957513 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a511056-1d71-4efb-9ea0-2e96a3f7cf5a None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:50.957791 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a511056-1d71-4efb-9ea0-2e96a3f7cf5a None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:50.958059 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a511056-1d71-4efb-9ea0-2e96a3f7cf5a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:50.959154 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a511056-1d71-4efb-9ea0-2e96a3f7cf5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:50.960590 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a511056-1d71-4efb-9ea0-2e96a3f7cf5a None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:50.962897 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 423/859] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Oct 31 12:21:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 31 12:21:50.968794 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9dd3d1-a7e9-4e9a-b664-1cffd8fb2206 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:51.064962 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9dd3d1-a7e9-4e9a-b664-1cffd8fb2206 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:51.097291 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84670d72-9a06-4096-8dba-27a26515f73f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:51.097831 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84670d72-9a06-4096-8dba-27a26515f73f None admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:51.098106 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84670d72-9a06-4096-8dba-27a26515f73f None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:51.098373 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84670d72-9a06-4096-8dba-27a26515f73f None admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:51.099004 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84670d72-9a06-4096-8dba-27a26515f73f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:51.100438 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84670d72-9a06-4096-8dba-27a26515f73f None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:51.126397 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 437/860] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Oct 31 12:21:50 2022] POST /identity/v3/projects => generated 311 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:21:53.267483 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fc0a4d-8eec-4333-9e4f-913878381dbb None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:53.268004 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fc0a4d-8eec-4333-9e4f-913878381dbb None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:53.268004 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fc0a4d-8eec-4333-9e4f-913878381dbb None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:53.269651 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 424/861] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:21:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:21:53.274934 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c90e140-ad76-4edd-a430-52e3a887fa74 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:53.275360 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c90e140-ad76-4edd-a430-52e3a887fa74 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:53.275511 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c90e140-ad76-4edd-a430-52e3a887fa74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:53.293927 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-0c90e140-ad76-4edd-a430-52e3a887fa74 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:21:53.295022 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-0c90e140-ad76-4edd-a430-52e3a887fa74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:21:53.297669 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:53.360517 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 438/862] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:21:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 31 12:21:53.631191 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fc0a4d-8eec-4333-9e4f-913878381dbb None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:53.729494 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fc0a4d-8eec-4333-9e4f-913878381dbb None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:53.763001 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69137712-a8a0-4ae5-9197-ba61a860fdae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:53.763648 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69137712-a8a0-4ae5-9197-ba61a860fdae None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:53.763926 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69137712-a8a0-4ae5-9197-ba61a860fdae None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:53.764192 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69137712-a8a0-4ae5-9197-ba61a860fdae None admin] PATH_INFO: `/v3/domains/default` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:53.765272 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69137712-a8a0-4ae5-9197-ba61a860fdae None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:53.766664 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69137712-a8a0-4ae5-9197-ba61a860fdae None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:53.768215 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 425/863] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Oct 31 12:21:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:21:53.774139 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c90e140-ad76-4edd-a430-52e3a887fa74 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:53.867534 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c90e140-ad76-4edd-a430-52e3a887fa74 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:53.900925 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fc3c03-bd02-45ea-a642-64e60a21928a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:53.901452 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fc3c03-bd02-45ea-a642-64e60a21928a None admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:53.901731 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fc3c03-bd02-45ea-a642-64e60a21928a None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:53.901997 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fc3c03-bd02-45ea-a642-64e60a21928a None admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:53.902632 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fc3c03-bd02-45ea-a642-64e60a21928a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:53.904063 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fc3c03-bd02-45ea-a642-64e60a21928a None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:53.915754 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:53.926578 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 439/864] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Oct 31 12:21:53 2022] POST /identity/v3/users => generated 286 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:21:56.074757 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a34d61-ece9-47fd-a708-b50374ae6d34 None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:56.075282 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a34d61-ece9-47fd-a708-b50374ae6d34 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:56.075380 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a34d61-ece9-47fd-a708-b50374ae6d34 None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:56.076657 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 426/865] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:21:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:21:56.081726 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecc9306-722e-4107-8898-625b56c86b8d None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:56.082110 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecc9306-722e-4107-8898-625b56c86b8d None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:56.082264 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecc9306-722e-4107-8898-625b56c86b8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:56.100226 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-fecc9306-722e-4107-8898-625b56c86b8d None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:21:56.101388 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-fecc9306-722e-4107-8898-625b56c86b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:21:56.165846 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 440/866] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:21:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 31 12:21:56.430693 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a34d61-ece9-47fd-a708-b50374ae6d34 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:56.434059 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:56.529336 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a34d61-ece9-47fd-a708-b50374ae6d34 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:56.561745 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385dfd35-b69e-46cb-9480-2de0297dc782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:56.562353 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385dfd35-b69e-46cb-9480-2de0297dc782 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:56.562626 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385dfd35-b69e-46cb-9480-2de0297dc782 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:56.562887 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385dfd35-b69e-46cb-9480-2de0297dc782 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:56.572691 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385dfd35-b69e-46cb-9480-2de0297dc782 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 31 12:21:56.577397 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-385dfd35-b69e-46cb-9480-2de0297dc782 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 31 12:21:56.577397 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:56.577397 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:21:56.577397 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:21:56.577397 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:21:56.577397 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:21:56.577397 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:21:56.577397 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:21:56.577397 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:21:56.577397 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:21:56.577397 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:21:56.577397 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:21:56.577397 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 31 12:21:56.577397 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 31 12:21:56.577397 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 31 12:21:56.577397 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 31 12:21:56.578541 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 31 12:21:56.578541 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 31 12:21:56.578541 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 31 12:21:56.578541 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 31 12:21:56.578541 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:21:56.578541 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:21:56.578541 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:21:56.578541 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:21:56.578541 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:21:56.578541 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:21:56.578541 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:21:56.578541 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:21:56.578541 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:21:56.578541 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:21:56.578541 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:21:56.578541 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:21:56.579685 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:21:56.579685 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:21:56.579685 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:21:56.579685 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:21:56.579685 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 31 12:21:56.579685 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 31 12:21:56.579685 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 31 12:21:56.579685 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 31 12:21:56.579685 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 31 12:21:56.579685 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 31 12:21:56.579685 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 31 12:21:56.579685 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:21:56.579685 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 427/867] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Oct 31 12:21:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:21:56.584388 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecc9306-722e-4107-8898-625b56c86b8d None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:56.679501 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecc9306-722e-4107-8898-625b56c86b8d None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:56.707418 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:56.716677 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9674e0c-0b1d-4bde-ba1f-8d416ef54846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:56.717279 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9674e0c-0b1d-4bde-ba1f-8d416ef54846 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:56.717554 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9674e0c-0b1d-4bde-ba1f-8d416ef54846 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:56.717821 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9674e0c-0b1d-4bde-ba1f-8d416ef54846 None admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:56.718472 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9674e0c-0b1d-4bde-ba1f-8d416ef54846 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:56.718772 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9674e0c-0b1d-4bde-ba1f-8d416ef54846 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:56.720154 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9674e0c-0b1d-4bde-ba1f-8d416ef54846 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:56.728557 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 441/868] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Oct 31 12:21:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:21:56.734367 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385dfd35-b69e-46cb-9480-2de0297dc782 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:56.795022 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:56.832770 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385dfd35-b69e-46cb-9480-2de0297dc782 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:56.866121 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c29dee-f14f-4d5f-ab30-6726d27d94fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:56.866733 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c29dee-f14f-4d5f-ab30-6726d27d94fb None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:56.867009 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c29dee-f14f-4d5f-ab30-6726d27d94fb None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:56.867307 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c29dee-f14f-4d5f-ab30-6726d27d94fb None admin] PATH_INFO: `/v3/users/88b9242c406e47039c49d524cfcbeb09` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:56.880448 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c29dee-f14f-4d5f-ab30-6726d27d94fb None admin] RBAC: Authorizing `identity:get_user(user_id=88b9242c406e47039c49d524cfcbeb09)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:56.881870 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c29dee-f14f-4d5f-ab30-6726d27d94fb None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:56.883531 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 428/869] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Oct 31 12:21:56 2022] GET /identity/v3/users/88b9242c406e47039c49d524cfcbeb09 => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:21:56.889287 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9674e0c-0b1d-4bde-ba1f-8d416ef54846 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:56.988542 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9674e0c-0b1d-4bde-ba1f-8d416ef54846 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:57.020718 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba951a30-3adf-4b09-a15a-b70fada1de82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:57.021549 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba951a30-3adf-4b09-a15a-b70fada1de82 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:57.021820 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba951a30-3adf-4b09-a15a-b70fada1de82 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:57.022085 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba951a30-3adf-4b09-a15a-b70fada1de82 None admin] PATH_INFO: `/v3/projects/a2ff7e7f98bf44b390a396e1098589ef` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:57.023221 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba951a30-3adf-4b09-a15a-b70fada1de82 None admin] RBAC: Authorizing `identity:get_project(project_id=a2ff7e7f98bf44b390a396e1098589ef)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:57.024782 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba951a30-3adf-4b09-a15a-b70fada1de82 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:57.026209 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 442/870] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Oct 31 12:21:56 2022] GET /identity/v3/projects/a2ff7e7f98bf44b390a396e1098589ef => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:21:57.032060 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c29dee-f14f-4d5f-ab30-6726d27d94fb None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:57.128385 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c29dee-f14f-4d5f-ab30-6726d27d94fb None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:57.147992 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:57.162131 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095b6675-d0b2-43c8-9b49-b26773483727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:57.162769 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095b6675-d0b2-43c8-9b49-b26773483727 None admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:57.163049 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095b6675-d0b2-43c8-9b49-b26773483727 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:57.163345 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095b6675-d0b2-43c8-9b49-b26773483727 None admin] PATH_INFO: `/v3/projects/a2ff7e7f98bf44b390a396e1098589ef/users/88b9242c406e47039c49d524cfcbeb09/roles/176c07b4fe3b4537bdfeb4be21b379c5` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:57.165332 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095b6675-d0b2-43c8-9b49-b26773483727 None admin] RBAC: Authorizing `identity:create_grant(project_id=a2ff7e7f98bf44b390a396e1098589ef, user_id=88b9242c406e47039c49d524cfcbeb09, role_id=176c07b4fe3b4537bdfeb4be21b379c5)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:57.166729 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095b6675-d0b2-43c8-9b49-b26773483727 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:57.178652 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 429/871] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Oct 31 12:21:57 2022] PUT /identity/v3/projects/a2ff7e7f98bf44b390a396e1098589ef/users/88b9242c406e47039c49d524cfcbeb09/roles/176c07b4fe3b4537bdfeb4be21b379c5 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:21:59.304166 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c4d676-a0c7-47c5-b9e0-a6333c21f4d9 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:59.304679 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c4d676-a0c7-47c5-b9e0-a6333c21f4d9 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:59.304679 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c4d676-a0c7-47c5-b9e0-a6333c21f4d9 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:59.306186 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 443/872] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:21:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:21:59.311484 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e779d988-0fda-468d-94b9-1469a51ddef4 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:59.311897 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e779d988-0fda-468d-94b9-1469a51ddef4 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:59.312017 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e779d988-0fda-468d-94b9-1469a51ddef4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:59.330171 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-e779d988-0fda-468d-94b9-1469a51ddef4 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:21:59.331367 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-e779d988-0fda-468d-94b9-1469a51ddef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:21:59.395180 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 430/873] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:21:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 31 12:21:59.659791 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c4d676-a0c7-47c5-b9e0-a6333c21f4d9 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:59.715804 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:59.758952 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c4d676-a0c7-47c5-b9e0-a6333c21f4d9 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:59.791908 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eb212a-869e-4826-b4fb-95dfda47054d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:59.792530 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eb212a-869e-4826-b4fb-95dfda47054d None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:59.792807 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eb212a-869e-4826-b4fb-95dfda47054d None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:59.793078 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eb212a-869e-4826-b4fb-95dfda47054d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:59.802607 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5eb212a-869e-4826-b4fb-95dfda47054d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 31 12:21:59.807666 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-f5eb212a-869e-4826-b4fb-95dfda47054d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 31 12:21:59.807666 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:21:59.807666 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:21:59.807666 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:21:59.807666 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:21:59.807666 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:21:59.807666 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:21:59.807666 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:21:59.807666 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:21:59.807666 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:21:59.807666 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:21:59.807666 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:21:59.807666 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 31 12:21:59.807666 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 31 12:21:59.807666 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 31 12:21:59.807666 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 31 12:21:59.808742 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 31 12:21:59.808742 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 31 12:21:59.808742 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 31 12:21:59.808742 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 31 12:21:59.808742 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:21:59.808742 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:21:59.808742 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:21:59.808742 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:21:59.808742 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:21:59.808742 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:21:59.808742 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:21:59.808742 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 31 12:21:59.808742 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:21:59.808742 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:21:59.808742 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:21:59.808742 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:21:59.809986 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:21:59.809986 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:21:59.809986 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:21:59.809986 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:21:59.809986 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 31 12:21:59.809986 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 31 12:21:59.809986 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 31 12:21:59.809986 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 31 12:21:59.809986 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 31 12:21:59.809986 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 31 12:21:59.809986 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 31 12:21:59.809986 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:21:59.809986 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 444/874] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Oct 31 12:21:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:21:59.814683 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e779d988-0fda-468d-94b9-1469a51ddef4 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:21:59.910237 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e779d988-0fda-468d-94b9-1469a51ddef4 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:21:59.921400 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:21:59.945928 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1cc8c2-b8dc-4b26-826f-aceb82a048c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:21:59.946523 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1cc8c2-b8dc-4b26-826f-aceb82a048c1 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:21:59.946795 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1cc8c2-b8dc-4b26-826f-aceb82a048c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:21:59.947061 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1cc8c2-b8dc-4b26-826f-aceb82a048c1 None admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:21:59.947771 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1cc8c2-b8dc-4b26-826f-aceb82a048c1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:21:59.948069 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1cc8c2-b8dc-4b26-826f-aceb82a048c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:21:59.949410 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1cc8c2-b8dc-4b26-826f-aceb82a048c1 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:21:59.957122 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 431/875] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Oct 31 12:21:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:21:59.963059 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eb212a-869e-4826-b4fb-95dfda47054d None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:22:00.059014 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eb212a-869e-4826-b4fb-95dfda47054d None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:22:00.074442 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:22:00.092049 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbc0866-de91-4a7a-bfdf-dbc78376994d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:22:00.092670 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbc0866-de91-4a7a-bfdf-dbc78376994d None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:22:00.092949 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbc0866-de91-4a7a-bfdf-dbc78376994d None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:22:00.093218 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbc0866-de91-4a7a-bfdf-dbc78376994d None admin] PATH_INFO: `/v3/users/88b9242c406e47039c49d524cfcbeb09` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:22:00.094340 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbc0866-de91-4a7a-bfdf-dbc78376994d None admin] RBAC: Authorizing `identity:get_user(user_id=88b9242c406e47039c49d524cfcbeb09)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:22:00.095788 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbc0866-de91-4a7a-bfdf-dbc78376994d None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:22:00.097343 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 445/876] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Oct 31 12:21:59 2022] GET /identity/v3/users/88b9242c406e47039c49d524cfcbeb09 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:22:00.103730 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1cc8c2-b8dc-4b26-826f-aceb82a048c1 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:22:00.316755 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1cc8c2-b8dc-4b26-826f-aceb82a048c1 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:22:00.348990 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c634c643-a96b-4821-850a-6a6198ca1e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:22:00.349571 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c634c643-a96b-4821-850a-6a6198ca1e1c None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:22:00.349846 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c634c643-a96b-4821-850a-6a6198ca1e1c None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:22:00.350112 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c634c643-a96b-4821-850a-6a6198ca1e1c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:22:00.389310 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:22:00.415095 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c634c643-a96b-4821-850a-6a6198ca1e1c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:22:00.416652 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c634c643-a96b-4821-850a-6a6198ca1e1c None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:22:00.510608 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 432/877] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Oct 31 12:22:00 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 31 12:22:00.516636 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbc0866-de91-4a7a-bfdf-dbc78376994d None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:22:00.612454 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbc0866-de91-4a7a-bfdf-dbc78376994d None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:22:00.645052 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996b22bc-b29b-4920-b521-f1acce6ebebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:22:00.645693 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996b22bc-b29b-4920-b521-f1acce6ebebd None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:22:00.645970 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996b22bc-b29b-4920-b521-f1acce6ebebd None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:22:00.646239 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996b22bc-b29b-4920-b521-f1acce6ebebd None admin] PATH_INFO: `/v3/projects/a2ff7e7f98bf44b390a396e1098589ef` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:22:00.647356 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996b22bc-b29b-4920-b521-f1acce6ebebd None admin] RBAC: Authorizing `identity:get_project(project_id=a2ff7e7f98bf44b390a396e1098589ef)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:22:00.648767 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996b22bc-b29b-4920-b521-f1acce6ebebd None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:22:00.650548 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 446/878] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Oct 31 12:22:00 2022] GET /identity/v3/projects/a2ff7e7f98bf44b390a396e1098589ef => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:22:00.656532 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c634c643-a96b-4821-850a-6a6198ca1e1c None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:22:00.725673 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:22:00.754475 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c634c643-a96b-4821-850a-6a6198ca1e1c None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:22:00.787302 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ae46da-f1c3-4029-bc91-769fe5388d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:22:00.787896 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ae46da-f1c3-4029-bc91-769fe5388d3d None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:22:00.788174 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ae46da-f1c3-4029-bc91-769fe5388d3d None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:22:00.788445 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ae46da-f1c3-4029-bc91-769fe5388d3d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:22:00.789156 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ae46da-f1c3-4029-bc91-769fe5388d3d None admin] RBAC: Adding query filter params (role.id=176c07b4fe3b4537bdfeb4be21b379c5, scope.project.id=a2ff7e7f98bf44b390a396e1098589ef, user.id=88b9242c406e47039c49d524cfcbeb09) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:22:00.790003 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ae46da-f1c3-4029-bc91-769fe5388d3d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:22:00.791511 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ae46da-f1c3-4029-bc91-769fe5388d3d None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:22:00.797076 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 433/879] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Oct 31 12:22:00 2022] GET /identity/v3/role_assignments?user.id=88b9242c406e47039c49d524cfcbeb09&scope.project.id=a2ff7e7f98bf44b390a396e1098589ef&role.id=176c07b4fe3b4537bdfeb4be21b379c5 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:22:02.926684 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba566796-a1cf-401a-b240-4aedfcc5fa2d None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:22:02.927370 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba566796-a1cf-401a-b240-4aedfcc5fa2d None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:22:02.927370 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba566796-a1cf-401a-b240-4aedfcc5fa2d None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:22:02.928579 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 447/880] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:22:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:22:02.934432 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b53c3-ed74-48ff-aa2e-a4c8fea8f850 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:22:02.934829 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b53c3-ed74-48ff-aa2e-a4c8fea8f850 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:22:02.934992 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b53c3-ed74-48ff-aa2e-a4c8fea8f850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:22:02.952745 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-7a9b53c3-ed74-48ff-aa2e-a4c8fea8f850 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:22:02.953924 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-7a9b53c3-ed74-48ff-aa2e-a4c8fea8f850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:22:03.017724 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 434/881] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:22:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 31 12:22:03.283214 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba566796-a1cf-401a-b240-4aedfcc5fa2d None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:22:03.335153 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:22:03.381190 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba566796-a1cf-401a-b240-4aedfcc5fa2d None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:22:03.414181 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8b4998-6aa0-4c94-813b-a5bda466a167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:22:03.415354 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8b4998-6aa0-4c94-813b-a5bda466a167 None admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:22:03.415632 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8b4998-6aa0-4c94-813b-a5bda466a167 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:22:03.415901 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8b4998-6aa0-4c94-813b-a5bda466a167 None admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:22:03.416560 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8b4998-6aa0-4c94-813b-a5bda466a167 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:22:03.417895 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8b4998-6aa0-4c94-813b-a5bda466a167 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:22:03.432313 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 448/882] 10.222.0.15 () {60 vars in 1138 bytes} [Mon Oct 31 12:22:03 2022] POST /identity/v3/domains => generated 267 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:22:05.581496 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c89b96-7b44-494c-b6b7-742d1bf650d0 None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:22:05.582033 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c89b96-7b44-494c-b6b7-742d1bf650d0 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:22:05.582033 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c89b96-7b44-494c-b6b7-742d1bf650d0 None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:22:05.583435 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 435/883] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:22:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:22:05.589138 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c146fd-c4cb-429a-b044-c7963239e75e None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:22:05.589569 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c146fd-c4cb-429a-b044-c7963239e75e None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:22:05.589665 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c146fd-c4cb-429a-b044-c7963239e75e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:22:05.608029 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-27c146fd-c4cb-429a-b044-c7963239e75e None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:22:05.609198 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-27c146fd-c4cb-429a-b044-c7963239e75e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:22:05.672994 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 449/884] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:22:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 31 12:22:05.946268 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c89b96-7b44-494c-b6b7-742d1bf650d0 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:22:06.010427 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:22:06.046703 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c89b96-7b44-494c-b6b7-742d1bf650d0 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:22:06.079637 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2323643-446e-48fc-91f2-db3cc0d8bb7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:22:06.080255 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2323643-446e-48fc-91f2-db3cc0d8bb7d None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:22:06.080534 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2323643-446e-48fc-91f2-db3cc0d8bb7d None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:22:06.080805 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2323643-446e-48fc-91f2-db3cc0d8bb7d None admin] PATH_INFO: `/v3/domains/ff436b68dc0640ed9b9e363150eeaaeb` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:22:06.091318 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2323643-446e-48fc-91f2-db3cc0d8bb7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=ff436b68dc0640ed9b9e363150eeaaeb)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:22:06.092743 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2323643-446e-48fc-91f2-db3cc0d8bb7d None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:22:06.094297 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 436/885] 10.222.0.15 () {56 vars in 1183 bytes} [Mon Oct 31 12:22:05 2022] GET /identity/v3/domains/ff436b68dc0640ed9b9e363150eeaaeb => generated 267 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:22:06.100122 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c146fd-c4cb-429a-b044-c7963239e75e None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:22:06.140971 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:22:06.195526 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c146fd-c4cb-429a-b044-c7963239e75e None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:22:06.228445 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc751e6a-b25b-4c26-81e1-441376b09392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:22:06.228979 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc751e6a-b25b-4c26-81e1-441376b09392 None admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:22:06.229253 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc751e6a-b25b-4c26-81e1-441376b09392 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:22:06.229520 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc751e6a-b25b-4c26-81e1-441376b09392 None admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:22:06.230144 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc751e6a-b25b-4c26-81e1-441376b09392 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:22:06.231591 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc751e6a-b25b-4c26-81e1-441376b09392 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:22:06.257302 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 450/886] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Oct 31 12:22:06 2022] POST /identity/v3/projects => generated 361 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:22:08.405382 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f09ee0-5418-4710-bad8-b79134f539ef None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:22:08.405905 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f09ee0-5418-4710-bad8-b79134f539ef None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:22:08.405905 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f09ee0-5418-4710-bad8-b79134f539ef None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:22:08.407199 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 437/887] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:22:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:22:08.412438 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4d75bf-5810-4fbe-a535-06bd1ae16a86 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:22:08.412824 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4d75bf-5810-4fbe-a535-06bd1ae16a86 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:22:08.412991 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4d75bf-5810-4fbe-a535-06bd1ae16a86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:22:08.431614 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-dd4d75bf-5810-4fbe-a535-06bd1ae16a86 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:22:08.432767 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-dd4d75bf-5810-4fbe-a535-06bd1ae16a86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:22:08.495764 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 451/888] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:22:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 31 12:22:08.768817 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f09ee0-5418-4710-bad8-b79134f539ef None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:22:08.871570 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f09ee0-5418-4710-bad8-b79134f539ef None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:22:08.896866 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:22:08.906695 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43b7db5-7467-4574-aecc-3d809cd13c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:22:08.907347 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43b7db5-7467-4574-aecc-3d809cd13c26 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:22:08.907631 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43b7db5-7467-4574-aecc-3d809cd13c26 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:22:08.907906 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43b7db5-7467-4574-aecc-3d809cd13c26 None admin] PATH_INFO: `/v3/domains/ff436b68dc0640ed9b9e363150eeaaeb` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:22:08.909005 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43b7db5-7467-4574-aecc-3d809cd13c26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=ff436b68dc0640ed9b9e363150eeaaeb)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:22:08.910411 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43b7db5-7467-4574-aecc-3d809cd13c26 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:22:08.911991 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 438/889] 10.222.0.15 () {56 vars in 1183 bytes} [Mon Oct 31 12:22:08 2022] GET /identity/v3/domains/ff436b68dc0640ed9b9e363150eeaaeb => generated 267 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:22:08.917771 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4d75bf-5810-4fbe-a535-06bd1ae16a86 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:22:08.941374 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:22:09.014893 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4d75bf-5810-4fbe-a535-06bd1ae16a86 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:22:09.049446 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e48a62-c32b-4363-87a8-bcc491d7e28b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:22:09.050052 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e48a62-c32b-4363-87a8-bcc491d7e28b None admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:22:09.050332 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e48a62-c32b-4363-87a8-bcc491d7e28b None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:22:09.050606 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e48a62-c32b-4363-87a8-bcc491d7e28b None admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:22:09.051284 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e48a62-c32b-4363-87a8-bcc491d7e28b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:22:09.052708 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e48a62-c32b-4363-87a8-bcc491d7e28b None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:22:09.074099 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 452/890] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Oct 31 12:22:08 2022] POST /identity/v3/users => generated 311 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 31 12:22:11.203583 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab51521e-91f1-4fdc-8a8b-46780d0eef48 None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:22:11.204123 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab51521e-91f1-4fdc-8a8b-46780d0eef48 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:22:11.204123 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab51521e-91f1-4fdc-8a8b-46780d0eef48 None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:22:11.205489 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 439/891] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:22:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:22:11.210482 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1ec707-1e8e-4cdd-9791-2d8dd51998a0 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:22:11.210862 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1ec707-1e8e-4cdd-9791-2d8dd51998a0 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:22:11.211021 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1ec707-1e8e-4cdd-9791-2d8dd51998a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:22:11.229322 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-4a1ec707-1e8e-4cdd-9791-2d8dd51998a0 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:22:11.230475 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-4a1ec707-1e8e-4cdd-9791-2d8dd51998a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:22:11.294456 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 453/892] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:22:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 31 12:22:11.559100 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab51521e-91f1-4fdc-8a8b-46780d0eef48 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:22:11.658133 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab51521e-91f1-4fdc-8a8b-46780d0eef48 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:22:11.690587 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f80b6d8-48d9-4d77-90e1-bbbc371651a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:22:11.691216 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f80b6d8-48d9-4d77-90e1-bbbc371651a2 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:22:11.691491 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f80b6d8-48d9-4d77-90e1-bbbc371651a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:22:11.691755 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f80b6d8-48d9-4d77-90e1-bbbc371651a2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:22:11.702696 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f80b6d8-48d9-4d77-90e1-bbbc371651a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 31 12:22:11.707449 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-6f80b6d8-48d9-4d77-90e1-bbbc371651a2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 31 12:22:11.707449 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:22:11.707449 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:22:11.707449 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:22:11.707449 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:22:11.707449 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:22:11.707449 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:22:11.707449 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:22:11.707449 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:22:11.707449 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:22:11.707449 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:22:11.707449 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:22:11.707449 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 31 12:22:11.707449 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 31 12:22:11.707449 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 31 12:22:11.707449 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 31 12:22:11.708579 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 31 12:22:11.708579 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 31 12:22:11.708579 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 31 12:22:11.708579 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 31 12:22:11.708579 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:22:11.708579 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:22:11.708579 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:22:11.708579 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:22:11.708579 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:22:11.708579 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:22:11.708579 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:22:11.708579 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:22:11.708579 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:22:11.708579 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:22:11.708579 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:22:11.708579 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:22:11.709481 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:22:11.709481 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:22:11.709481 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:22:11.709481 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:22:11.709481 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 31 12:22:11.709481 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 31 12:22:11.709481 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 31 12:22:11.709481 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 31 12:22:11.709481 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 31 12:22:11.709481 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 31 12:22:11.709481 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 31 12:22:11.709481 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:22:11.709481 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 440/893] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Oct 31 12:22:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:22:11.714660 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1ec707-1e8e-4cdd-9791-2d8dd51998a0 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:22:11.725940 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:22:11.811502 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1ec707-1e8e-4cdd-9791-2d8dd51998a0 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:22:11.843739 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8564597-a9bd-4c36-af03-97637d20fc2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:22:11.844329 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8564597-a9bd-4c36-af03-97637d20fc2e None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:22:11.844604 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8564597-a9bd-4c36-af03-97637d20fc2e None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:22:11.844870 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8564597-a9bd-4c36-af03-97637d20fc2e None admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:22:11.845527 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8564597-a9bd-4c36-af03-97637d20fc2e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:22:11.845827 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8564597-a9bd-4c36-af03-97637d20fc2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:22:11.847188 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8564597-a9bd-4c36-af03-97637d20fc2e None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:22:11.855539 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 454/894] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Oct 31 12:22:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:22:11.861253 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f80b6d8-48d9-4d77-90e1-bbbc371651a2 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:22:11.914100 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:22:11.957580 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f80b6d8-48d9-4d77-90e1-bbbc371651a2 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:22:11.989357 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb6f11c-888c-4500-a72d-6d6b236d7095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:22:11.989971 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb6f11c-888c-4500-a72d-6d6b236d7095 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:22:11.990246 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb6f11c-888c-4500-a72d-6d6b236d7095 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:22:11.990514 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb6f11c-888c-4500-a72d-6d6b236d7095 None admin] PATH_INFO: `/v3/users/2efb751eb4e74b21b20eb024c4f4094e` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:22:12.003469 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb6f11c-888c-4500-a72d-6d6b236d7095 None admin] RBAC: Authorizing `identity:get_user(user_id=2efb751eb4e74b21b20eb024c4f4094e)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:22:12.004874 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb6f11c-888c-4500-a72d-6d6b236d7095 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:22:12.006424 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 441/895] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Oct 31 12:22:11 2022] GET /identity/v3/users/2efb751eb4e74b21b20eb024c4f4094e => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:22:12.012160 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8564597-a9bd-4c36-af03-97637d20fc2e None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:22:12.082490 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:22:12.110919 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8564597-a9bd-4c36-af03-97637d20fc2e None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:22:12.143269 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fffec2f-38c5-4247-8438-1c5a99bbbc8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:22:12.143887 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fffec2f-38c5-4247-8438-1c5a99bbbc8e None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:22:12.144412 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fffec2f-38c5-4247-8438-1c5a99bbbc8e None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:22:12.144679 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fffec2f-38c5-4247-8438-1c5a99bbbc8e None admin] PATH_INFO: `/v3/projects/83aa3a4019734f9498295750a5ba3b85` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:22:12.145768 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fffec2f-38c5-4247-8438-1c5a99bbbc8e None admin] RBAC: Authorizing `identity:get_project(project_id=83aa3a4019734f9498295750a5ba3b85)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:22:12.147194 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fffec2f-38c5-4247-8438-1c5a99bbbc8e None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:22:12.149385 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 455/896] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Oct 31 12:22:12 2022] GET /identity/v3/projects/83aa3a4019734f9498295750a5ba3b85 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:22:12.155364 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb6f11c-888c-4500-a72d-6d6b236d7095 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:22:12.250356 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb6f11c-888c-4500-a72d-6d6b236d7095 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:22:12.262891 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:22:12.283669 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f8709d-1c61-4a83-9782-3dbd9e065b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:22:12.284310 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f8709d-1c61-4a83-9782-3dbd9e065b67 None admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:22:12.284588 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f8709d-1c61-4a83-9782-3dbd9e065b67 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:22:12.284857 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f8709d-1c61-4a83-9782-3dbd9e065b67 None admin] PATH_INFO: `/v3/projects/83aa3a4019734f9498295750a5ba3b85/users/2efb751eb4e74b21b20eb024c4f4094e/roles/176c07b4fe3b4537bdfeb4be21b379c5` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:22:12.286720 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f8709d-1c61-4a83-9782-3dbd9e065b67 None admin] RBAC: Authorizing `identity:create_grant(project_id=83aa3a4019734f9498295750a5ba3b85, user_id=2efb751eb4e74b21b20eb024c4f4094e, role_id=176c07b4fe3b4537bdfeb4be21b379c5)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:22:12.288151 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f8709d-1c61-4a83-9782-3dbd9e065b67 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:22:12.301682 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 442/897] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Oct 31 12:22:12 2022] PUT /identity/v3/projects/83aa3a4019734f9498295750a5ba3b85/users/2efb751eb4e74b21b20eb024c4f4094e/roles/176c07b4fe3b4537bdfeb4be21b379c5 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:22:14.411425 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e0b97e-d4f0-401b-a720-465f0e67357f None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:22:14.411924 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e0b97e-d4f0-401b-a720-465f0e67357f None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:22:14.411924 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e0b97e-d4f0-401b-a720-465f0e67357f None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:22:14.413195 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 456/898] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:22:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:22:14.418635 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1ded2b-9ccb-4032-8d94-cd7b72ed46e8 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:22:14.419054 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1ded2b-9ccb-4032-8d94-cd7b72ed46e8 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:22:14.419151 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1ded2b-9ccb-4032-8d94-cd7b72ed46e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:22:14.437496 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-8a1ded2b-9ccb-4032-8d94-cd7b72ed46e8 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:22:14.438659 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-8a1ded2b-9ccb-4032-8d94-cd7b72ed46e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:22:14.502628 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 443/899] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:22:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 31 12:22:14.778954 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e0b97e-d4f0-401b-a720-465f0e67357f None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:22:14.876647 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e0b97e-d4f0-401b-a720-465f0e67357f None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:22:14.908976 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc88d3f9-3748-44ec-ac28-dbef86090be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:22:14.909585 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc88d3f9-3748-44ec-ac28-dbef86090be2 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:22:14.909861 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc88d3f9-3748-44ec-ac28-dbef86090be2 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:22:14.910155 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc88d3f9-3748-44ec-ac28-dbef86090be2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:22:14.917997 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:22:14.921053 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc88d3f9-3748-44ec-ac28-dbef86090be2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 31 12:22:14.926293 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-fc88d3f9-3748-44ec-ac28-dbef86090be2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 31 12:22:14.926293 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:22:14.926293 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:22:14.926293 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:22:14.926293 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:22:14.926293 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:22:14.926293 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:22:14.926293 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:22:14.926293 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:22:14.926293 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:22:14.926293 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:22:14.926293 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:22:14.926293 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 31 12:22:14.926293 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 31 12:22:14.926293 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 31 12:22:14.926293 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 31 12:22:14.927415 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 31 12:22:14.927415 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 31 12:22:14.927415 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 31 12:22:14.927415 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 31 12:22:14.927415 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:22:14.927415 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:22:14.927415 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:22:14.927415 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:22:14.927415 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:22:14.927415 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:22:14.927415 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:22:14.927415 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 31 12:22:14.927415 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:22:14.927415 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:22:14.927415 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:22:14.927415 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:22:14.928295 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:22:14.928295 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:22:14.928295 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:22:14.928295 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:22:14.928295 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 31 12:22:14.928295 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 31 12:22:14.928295 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 31 12:22:14.928295 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 31 12:22:14.928295 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 31 12:22:14.928295 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 31 12:22:14.928295 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 31 12:22:14.928295 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:22:14.928295 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 457/900] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Oct 31 12:22:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:22:14.933703 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1ded2b-9ccb-4032-8d94-cd7b72ed46e8 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:22:15.032267 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1ded2b-9ccb-4032-8d94-cd7b72ed46e8 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:22:15.033699 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:22:15.066354 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6687f99-5488-4c6b-9042-42a087a485cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:22:15.066946 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6687f99-5488-4c6b-9042-42a087a485cf None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:22:15.067262 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6687f99-5488-4c6b-9042-42a087a485cf None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:22:15.067530 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6687f99-5488-4c6b-9042-42a087a485cf None admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:22:15.068174 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6687f99-5488-4c6b-9042-42a087a485cf None admin] RBAC: Adding query filter params (name=admin) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:22:15.068471 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6687f99-5488-4c6b-9042-42a087a485cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:22:15.069814 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6687f99-5488-4c6b-9042-42a087a485cf None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:22:15.077986 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 444/901] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Oct 31 12:22:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:22:15.083980 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc88d3f9-3748-44ec-ac28-dbef86090be2 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:22:15.178755 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc88d3f9-3748-44ec-ac28-dbef86090be2 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:22:15.212149 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8026ebad-dff9-4347-aac6-2ec759842636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:22:15.212769 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8026ebad-dff9-4347-aac6-2ec759842636 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:22:15.213045 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8026ebad-dff9-4347-aac6-2ec759842636 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:22:15.213313 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8026ebad-dff9-4347-aac6-2ec759842636 None admin] PATH_INFO: `/v3/users/2efb751eb4e74b21b20eb024c4f4094e` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:22:15.214408 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8026ebad-dff9-4347-aac6-2ec759842636 None admin] RBAC: Authorizing `identity:get_user(user_id=2efb751eb4e74b21b20eb024c4f4094e)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:22:15.215856 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8026ebad-dff9-4347-aac6-2ec759842636 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:22:15.217401 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 458/902] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Oct 31 12:22:15 2022] GET /identity/v3/users/2efb751eb4e74b21b20eb024c4f4094e => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:22:15.223405 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6687f99-5488-4c6b-9042-42a087a485cf None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:22:15.323749 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6687f99-5488-4c6b-9042-42a087a485cf None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:22:15.357284 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49fb38b-7c9a-4261-b343-19d3e5812bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:22:15.357871 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49fb38b-7c9a-4261-b343-19d3e5812bde None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:22:15.358142 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49fb38b-7c9a-4261-b343-19d3e5812bde None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:22:15.358406 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49fb38b-7c9a-4261-b343-19d3e5812bde None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:22:15.386926 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:22:15.423002 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49fb38b-7c9a-4261-b343-19d3e5812bde None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:22:15.424445 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49fb38b-7c9a-4261-b343-19d3e5812bde None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:22:15.518409 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 445/903] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Oct 31 12:22:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 31 12:22:15.525426 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8026ebad-dff9-4347-aac6-2ec759842636 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:22:15.589898 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:22:15.621935 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8026ebad-dff9-4347-aac6-2ec759842636 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:22:15.655423 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ab93da-aefd-4786-b7d9-c087ad5cba28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:22:15.656038 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ab93da-aefd-4786-b7d9-c087ad5cba28 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:22:15.656315 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ab93da-aefd-4786-b7d9-c087ad5cba28 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:22:15.656583 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ab93da-aefd-4786-b7d9-c087ad5cba28 None admin] PATH_INFO: `/v3/projects/83aa3a4019734f9498295750a5ba3b85` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:22:15.657671 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ab93da-aefd-4786-b7d9-c087ad5cba28 None admin] RBAC: Authorizing `identity:get_project(project_id=83aa3a4019734f9498295750a5ba3b85)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:22:15.659078 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ab93da-aefd-4786-b7d9-c087ad5cba28 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:22:15.660693 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 459/904] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Oct 31 12:22:15 2022] GET /identity/v3/projects/83aa3a4019734f9498295750a5ba3b85 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:22:15.666770 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49fb38b-7c9a-4261-b343-19d3e5812bde None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:22:15.725741 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:22:15.762081 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49fb38b-7c9a-4261-b343-19d3e5812bde None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:22:15.794810 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86217880-5768-49cf-9768-f9857509e3fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:22:15.795416 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86217880-5768-49cf-9768-f9857509e3fb None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:22:15.795690 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86217880-5768-49cf-9768-f9857509e3fb None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:22:15.795954 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86217880-5768-49cf-9768-f9857509e3fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:22:15.796649 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86217880-5768-49cf-9768-f9857509e3fb None admin] RBAC: Adding query filter params (role.id=176c07b4fe3b4537bdfeb4be21b379c5, scope.project.id=83aa3a4019734f9498295750a5ba3b85, user.id=2efb751eb4e74b21b20eb024c4f4094e) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:22:15.796954 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86217880-5768-49cf-9768-f9857509e3fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:22:15.798345 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86217880-5768-49cf-9768-f9857509e3fb None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:22:15.803882 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 446/905] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Oct 31 12:22:15 2022] GET /identity/v3/role_assignments?user.id=2efb751eb4e74b21b20eb024c4f4094e&scope.project.id=83aa3a4019734f9498295750a5ba3b85&role.id=176c07b4fe3b4537bdfeb4be21b379c5 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:22:25.651381 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=32 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:22:25.801369 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=36 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:23:11.347994 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7982748f-2235-4057-994d-3673e6970290 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:23:11.348615 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7982748f-2235-4057-994d-3673e6970290 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:23:11.348615 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7982748f-2235-4057-994d-3673e6970290 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:23:11.350826 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 460/906] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:23:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:23:11.356612 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1db903-df2c-4b33-af50-c00fff665424 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:23:11.357144 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1db903-df2c-4b33-af50-c00fff665424 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:23:11.357144 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1db903-df2c-4b33-af50-c00fff665424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:23:11.376812 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-fc1db903-df2c-4b33-af50-c00fff665424 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:23:11.378017 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-fc1db903-df2c-4b33-af50-c00fff665424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:23:11.471403 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 447/907] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:23:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 31 12:23:11.738061 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7982748f-2235-4057-994d-3673e6970290 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:23:11.854796 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7982748f-2235-4057-994d-3673e6970290 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:23:11.897634 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681761d8-9c44-4c99-85fe-2a4c6d2c30dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:23:11.898329 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681761d8-9c44-4c99-85fe-2a4c6d2c30dc None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:23:11.898634 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681761d8-9c44-4c99-85fe-2a4c6d2c30dc None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:23:11.898909 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681761d8-9c44-4c99-85fe-2a4c6d2c30dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:23:11.910961 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681761d8-9c44-4c99-85fe-2a4c6d2c30dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:23:11.912612 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681761d8-9c44-4c99-85fe-2a4c6d2c30dc None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:23:11.928499 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-681761d8-9c44-4c99-85fe-2a4c6d2c30dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:23:11.928499 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:23:11.928499 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:23:11.928499 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:23:11.928499 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:23:11.928499 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:23:11.928499 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:23:11.928499 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:23:11.928499 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:23:11.928499 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:23:11.928499 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:23:11.928499 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:23:11.928499 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 31 12:23:11.928499 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 31 12:23:11.928499 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 31 12:23:11.928499 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 31 12:23:11.929577 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:23:11.929577 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:23:11.929577 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:23:11.929577 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:23:11.929577 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:23:11.929577 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:23:11.929577 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:23:11.929577 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 31 12:23:11.929577 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:23:11.929577 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:23:11.929577 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:23:11.929577 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:23:11.929577 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:23:11.929577 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:23:11.929577 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:23:11.929577 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:23:11.929577 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 31 12:23:11.930552 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 31 12:23:11.930552 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:23:11.930552 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:23:11.930552 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 461/908] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Oct 31 12:23:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:23:11.936505 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1db903-df2c-4b33-af50-c00fff665424 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:23:12.057784 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1db903-df2c-4b33-af50-c00fff665424 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:23:12.094462 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:23:12.102203 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654e655e-3821-4ea0-ac51-389b425142ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:23:12.102781 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654e655e-3821-4ea0-ac51-389b425142ef None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:23:12.103060 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654e655e-3821-4ea0-ac51-389b425142ef None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:23:12.103374 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654e655e-3821-4ea0-ac51-389b425142ef None admin] PATH_INFO: `/v3/domains` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:23:12.104018 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654e655e-3821-4ea0-ac51-389b425142ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:23:12.104317 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654e655e-3821-4ea0-ac51-389b425142ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:23:12.105823 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654e655e-3821-4ea0-ac51-389b425142ef None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:23:12.123213 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 448/909] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Oct 31 12:23:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 31 12:23:12.124588 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681761d8-9c44-4c99-85fe-2a4c6d2c30dc None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:23:12.190854 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:23:12.250337 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681761d8-9c44-4c99-85fe-2a4c6d2c30dc None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:23:12.287812 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c5a3e1-67ab-4495-a947-d7726bb7c775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:23:12.288419 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c5a3e1-67ab-4495-a947-d7726bb7c775 None admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:23:12.288686 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c5a3e1-67ab-4495-a947-d7726bb7c775 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:23:12.288915 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c5a3e1-67ab-4495-a947-d7726bb7c775 None admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:23:12.289546 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c5a3e1-67ab-4495-a947-d7726bb7c775 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:23:12.290959 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c5a3e1-67ab-4495-a947-d7726bb7c775 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:23:12.335200 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 462/910] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Oct 31 12:23:12 2022] POST /identity/v3/users => generated 251 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:23:14.697762 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991f65b6-a06b-45be-9d91-7611f16582a7 None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:23:14.700315 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991f65b6-a06b-45be-9d91-7611f16582a7 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:23:14.700315 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991f65b6-a06b-45be-9d91-7611f16582a7 None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:23:14.700836 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 449/911] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:23:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:23:14.705852 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831af81e-01ab-4606-ba5b-c0f0579081f0 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:23:14.706254 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831af81e-01ab-4606-ba5b-c0f0579081f0 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:23:14.706454 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831af81e-01ab-4606-ba5b-c0f0579081f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:23:14.729173 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-831af81e-01ab-4606-ba5b-c0f0579081f0 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:23:14.730330 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-831af81e-01ab-4606-ba5b-c0f0579081f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:23:14.809053 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 463/912] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:23:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 31 12:23:15.098563 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991f65b6-a06b-45be-9d91-7611f16582a7 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:23:15.227573 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991f65b6-a06b-45be-9d91-7611f16582a7 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:23:15.275592 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e5ee79-6263-4dc7-b76b-facabfdee2d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:23:15.276231 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e5ee79-6263-4dc7-b76b-facabfdee2d3 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:23:15.276492 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e5ee79-6263-4dc7-b76b-facabfdee2d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:23:15.276767 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e5ee79-6263-4dc7-b76b-facabfdee2d3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:23:15.290187 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e5ee79-6263-4dc7-b76b-facabfdee2d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 31 12:23:15.295251 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-d0e5ee79-6263-4dc7-b76b-facabfdee2d3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 31 12:23:15.295251 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:23:15.295251 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:23:15.295251 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:23:15.295251 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:23:15.295251 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:23:15.295251 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:23:15.295251 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:23:15.295251 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:23:15.295251 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:23:15.295251 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:23:15.295251 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:23:15.295251 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 31 12:23:15.295251 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 31 12:23:15.295251 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 31 12:23:15.295251 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 31 12:23:15.296316 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 31 12:23:15.296316 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 31 12:23:15.296316 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 31 12:23:15.296316 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 31 12:23:15.296316 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:23:15.296316 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:23:15.296316 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:23:15.296316 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:23:15.296316 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:23:15.296316 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:23:15.296316 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:23:15.296316 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:23:15.296316 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:23:15.296316 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:23:15.296316 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:23:15.296316 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:23:15.297534 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:23:15.297534 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:23:15.297534 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:23:15.297534 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:23:15.297534 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 31 12:23:15.297534 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 31 12:23:15.297534 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 31 12:23:15.297534 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 31 12:23:15.297534 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 31 12:23:15.297534 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 31 12:23:15.297534 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 31 12:23:15.297534 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:23:15.297534 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 450/913] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Oct 31 12:23:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:23:15.302621 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831af81e-01ab-4606-ba5b-c0f0579081f0 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:23:15.350916 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:23:15.422815 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831af81e-01ab-4606-ba5b-c0f0579081f0 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:23:15.463572 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed4807d-7780-4e09-a981-e14af0f90818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:23:15.464181 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed4807d-7780-4e09-a981-e14af0f90818 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:23:15.464453 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed4807d-7780-4e09-a981-e14af0f90818 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:23:15.464720 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed4807d-7780-4e09-a981-e14af0f90818 None admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:23:15.465483 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed4807d-7780-4e09-a981-e14af0f90818 None admin] RBAC: Adding query filter params (name=service) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:23:15.465748 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed4807d-7780-4e09-a981-e14af0f90818 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:23:15.467214 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed4807d-7780-4e09-a981-e14af0f90818 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:23:15.481716 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 464/914] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Oct 31 12:23:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 31 12:23:15.482865 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e5ee79-6263-4dc7-b76b-facabfdee2d3 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:23:15.546402 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:23:15.611274 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e5ee79-6263-4dc7-b76b-facabfdee2d3 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:23:15.652712 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c057cdf6-b4fb-4647-be2f-ce371dc9099b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:23:15.654173 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c057cdf6-b4fb-4647-be2f-ce371dc9099b None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:23:15.654173 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c057cdf6-b4fb-4647-be2f-ce371dc9099b None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:23:15.654173 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c057cdf6-b4fb-4647-be2f-ce371dc9099b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:23:15.666739 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c057cdf6-b4fb-4647-be2f-ce371dc9099b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:23:15.668349 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c057cdf6-b4fb-4647-be2f-ce371dc9099b None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:23:15.685245 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-c057cdf6-b4fb-4647-be2f-ce371dc9099b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:23:15.685245 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:23:15.685245 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:23:15.685245 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:23:15.685245 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:23:15.685245 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:23:15.685245 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:23:15.685245 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:23:15.685245 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:23:15.685245 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:23:15.685245 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:23:15.685245 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:23:15.685245 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 31 12:23:15.685245 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 31 12:23:15.685245 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 31 12:23:15.685245 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 31 12:23:15.686284 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:23:15.686284 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:23:15.686284 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:23:15.686284 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:23:15.686284 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:23:15.686284 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:23:15.686284 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:23:15.686284 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:23:15.686284 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:23:15.686284 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:23:15.686284 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:23:15.686284 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:23:15.686284 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:23:15.686284 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:23:15.686284 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:23:15.686284 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:23:15.686284 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 31 12:23:15.687782 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 31 12:23:15.687782 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:23:15.687782 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:23:15.687782 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 451/915] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Oct 31 12:23:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 31 12:23:15.692779 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed4807d-7780-4e09-a981-e14af0f90818 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:23:15.820709 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:23:15.822819 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed4807d-7780-4e09-a981-e14af0f90818 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:23:15.867798 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b063d64-4a47-4a85-b1cb-26e76f0da7bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:23:15.868419 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b063d64-4a47-4a85-b1cb-26e76f0da7bb None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:23:15.868698 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b063d64-4a47-4a85-b1cb-26e76f0da7bb None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:23:15.869067 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b063d64-4a47-4a85-b1cb-26e76f0da7bb None admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:23:15.869642 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b063d64-4a47-4a85-b1cb-26e76f0da7bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:23:15.870018 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b063d64-4a47-4a85-b1cb-26e76f0da7bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:23:15.871602 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b063d64-4a47-4a85-b1cb-26e76f0da7bb None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:23:15.885762 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 465/916] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Oct 31 12:23:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:23:15.891788 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c057cdf6-b4fb-4647-be2f-ce371dc9099b None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:23:16.020941 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c057cdf6-b4fb-4647-be2f-ce371dc9099b None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:23:16.028307 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:23:16.064311 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf6f9b4-a127-4d89-b9dd-5d14429279ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:23:16.064943 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf6f9b4-a127-4d89-b9dd-5d14429279ed None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:23:16.065203 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf6f9b4-a127-4d89-b9dd-5d14429279ed None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:23:16.065567 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf6f9b4-a127-4d89-b9dd-5d14429279ed None admin] PATH_INFO: `/v3/users/placement` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:23:16.075239 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf6f9b4-a127-4d89-b9dd-5d14429279ed None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:23:16.076635 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf6f9b4-a127-4d89-b9dd-5d14429279ed None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:23:16.088991 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-8bf6f9b4-a127-4d89-b9dd-5d14429279ed None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 31 12:23:16.088991 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:23:16.088991 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:23:16.088991 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:23:16.088991 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:23:16.088991 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:23:16.088991 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:23:16.088991 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:23:16.088991 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:23:16.088991 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:23:16.088991 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:23:16.088991 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:23:16.088991 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 31 12:23:16.088991 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 31 12:23:16.088991 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 31 12:23:16.088991 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 31 12:23:16.090199 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:23:16.090199 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:23:16.090199 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 31 12:23:16.090199 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 31 12:23:16.090199 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 31 12:23:16.090199 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 31 12:23:16.090199 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:23:16.090199 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:23:16.090199 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:23:16.090199 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:23:16.090199 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:23:16.090199 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:23:16.090199 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:23:16.090199 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:23:16.090199 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:23:16.090199 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:23:16.090199 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:23:16.090199 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:23:16.091419 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:23:16.091419 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:23:16.091419 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 31 12:23:16.091419 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 31 12:23:16.091419 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 31 12:23:16.091419 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 31 12:23:16.091419 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 31 12:23:16.091419 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 31 12:23:16.091419 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 31 12:23:16.091419 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:23:16.091419 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 452/917] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Oct 31 12:23:15 2022] GET /identity/v3/users/placement => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:23:16.098028 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b063d64-4a47-4a85-b1cb-26e76f0da7bb None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:23:16.210983 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b063d64-4a47-4a85-b1cb-26e76f0da7bb None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:23:16.247696 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884be220-8aeb-45d0-b738-9ff3fe0127f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:23:16.248456 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884be220-8aeb-45d0-b738-9ff3fe0127f6 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:23:16.248738 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884be220-8aeb-45d0-b738-9ff3fe0127f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:23:16.249064 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884be220-8aeb-45d0-b738-9ff3fe0127f6 None admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:23:16.249903 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884be220-8aeb-45d0-b738-9ff3fe0127f6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:23:16.250291 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884be220-8aeb-45d0-b738-9ff3fe0127f6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:23:16.251829 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884be220-8aeb-45d0-b738-9ff3fe0127f6 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:23:16.276711 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 466/918] 10.222.0.15 () {56 vars in 1143 bytes} [Mon Oct 31 12:23:16 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 31 12:23:16.278198 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf6f9b4-a127-4d89-b9dd-5d14429279ed None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:23:16.388014 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf6f9b4-a127-4d89-b9dd-5d14429279ed None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:23:16.425307 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c637abb1-ed53-412b-814e-7f4014749715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:23:16.425934 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c637abb1-ed53-412b-814e-7f4014749715 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:23:16.426208 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c637abb1-ed53-412b-814e-7f4014749715 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:23:16.426473 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c637abb1-ed53-412b-814e-7f4014749715 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:23:16.438314 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c637abb1-ed53-412b-814e-7f4014749715 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:23:16.439811 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c637abb1-ed53-412b-814e-7f4014749715 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:23:16.455835 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-c637abb1-ed53-412b-814e-7f4014749715 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:23:16.455835 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:23:16.455835 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:23:16.455835 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:23:16.455835 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:23:16.455835 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:23:16.455835 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:23:16.455835 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:23:16.455835 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:23:16.455835 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:23:16.455835 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:23:16.455835 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:23:16.455835 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 31 12:23:16.455835 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 31 12:23:16.455835 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 31 12:23:16.455835 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 31 12:23:16.456851 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:23:16.456851 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:23:16.456851 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:23:16.456851 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:23:16.456851 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:23:16.456851 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:23:16.456851 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:23:16.456851 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:23:16.456851 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:23:16.456851 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:23:16.456851 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:23:16.456851 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:23:16.456851 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:23:16.456851 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:23:16.456851 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:23:16.456851 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:23:16.456851 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 31 12:23:16.457753 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 31 12:23:16.457753 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:23:16.457753 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:23:16.457753 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 453/919] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Oct 31 12:23:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 31 12:23:16.462763 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884be220-8aeb-45d0-b738-9ff3fe0127f6 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:23:16.482086 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:23:16.579464 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884be220-8aeb-45d0-b738-9ff3fe0127f6 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:23:16.619510 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1968b2-112e-4ab4-80f1-61a1c17290ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:23:16.620122 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1968b2-112e-4ab4-80f1-61a1c17290ab None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:23:16.620400 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1968b2-112e-4ab4-80f1-61a1c17290ab None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:23:16.620824 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1968b2-112e-4ab4-80f1-61a1c17290ab None admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:23:16.621770 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1968b2-112e-4ab4-80f1-61a1c17290ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:23:16.622052 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1968b2-112e-4ab4-80f1-61a1c17290ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:23:16.623634 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1968b2-112e-4ab4-80f1-61a1c17290ab None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:23:16.638759 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 467/920] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Oct 31 12:23:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 31 12:23:16.644314 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c637abb1-ed53-412b-814e-7f4014749715 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:23:16.665384 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:23:16.782281 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c637abb1-ed53-412b-814e-7f4014749715 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:23:16.831490 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb605167-02ad-492b-b429-2a41fc0184cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:23:16.832488 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb605167-02ad-492b-b429-2a41fc0184cd None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:23:16.832763 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb605167-02ad-492b-b429-2a41fc0184cd None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:23:16.833031 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb605167-02ad-492b-b429-2a41fc0184cd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:23:16.840640 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb605167-02ad-492b-b429-2a41fc0184cd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:23:16.842081 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb605167-02ad-492b-b429-2a41fc0184cd None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:23:16.853887 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-eb605167-02ad-492b-b429-2a41fc0184cd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 31 12:23:16.853887 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:23:16.853887 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:23:16.853887 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:23:16.853887 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:23:16.853887 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:23:16.853887 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:23:16.853887 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:23:16.853887 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:23:16.853887 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:23:16.853887 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:23:16.853887 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:23:16.853887 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 31 12:23:16.853887 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 31 12:23:16.853887 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 31 12:23:16.854832 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 31 12:23:16.854832 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:23:16.854832 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:23:16.854832 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:23:16.854832 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:23:16.854832 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:23:16.854832 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:23:16.854832 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:23:16.854832 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:23:16.854832 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:23:16.854832 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:23:16.854832 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:23:16.854832 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:23:16.854832 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:23:16.854832 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:23:16.854832 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:23:16.854832 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:23:16.862516 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 31 12:23:16.862516 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 31 12:23:16.862516 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 31 12:23:16.862516 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 31 12:23:16.862516 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 31 12:23:16.862516 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 31 12:23:16.862516 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 31 12:23:16.862516 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:23:16.862516 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 454/921] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Oct 31 12:23:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:23:16.863033 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1968b2-112e-4ab4-80f1-61a1c17290ab None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:23:16.951220 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:23:16.976108 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1968b2-112e-4ab4-80f1-61a1c17290ab None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:23:17.030905 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9a7a27-dc56-4c20-9380-b8a6afc8590b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:23:17.031816 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a7a27-dc56-4c20-9380-b8a6afc8590b None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:23:17.032182 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a7a27-dc56-4c20-9380-b8a6afc8590b None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:23:17.032486 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a7a27-dc56-4c20-9380-b8a6afc8590b None admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:23:17.033322 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9a7a27-dc56-4c20-9380-b8a6afc8590b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:23:17.033625 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9a7a27-dc56-4c20-9380-b8a6afc8590b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:23:17.035460 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9a7a27-dc56-4c20-9380-b8a6afc8590b None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:23:17.050088 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 468/922] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Oct 31 12:23:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:23:17.057537 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb605167-02ad-492b-b429-2a41fc0184cd None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:23:17.151970 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:23:17.181947 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb605167-02ad-492b-b429-2a41fc0184cd None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:23:17.219875 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53af0dfb-7916-4f14-85cd-750f274fcd73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:23:17.220459 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53af0dfb-7916-4f14-85cd-750f274fcd73 None admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:23:17.220779 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53af0dfb-7916-4f14-85cd-750f274fcd73 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:23:17.221113 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53af0dfb-7916-4f14-85cd-750f274fcd73 None admin] PATH_INFO: `/v3/projects/2c32444cef584bce966b38203ce128b2/users/9e7777495d7a406c8c50b1ad52e47263/roles/9674a2c5acc74dfeb93cfd67dfd890dd` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:23:17.238100 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53af0dfb-7916-4f14-85cd-750f274fcd73 None admin] RBAC: Authorizing `identity:create_grant(project_id=2c32444cef584bce966b38203ce128b2, user_id=9e7777495d7a406c8c50b1ad52e47263, role_id=9674a2c5acc74dfeb93cfd67dfd890dd)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:23:17.239574 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53af0dfb-7916-4f14-85cd-750f274fcd73 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:23:17.253111 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 455/923] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Oct 31 12:23:17 2022] PUT /identity/v3/projects/2c32444cef584bce966b38203ce128b2/users/9e7777495d7a406c8c50b1ad52e47263/roles/9674a2c5acc74dfeb93cfd67dfd890dd => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:23:19.595350 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c541dcc-a5af-40f0-b765-66ea04973ebe None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:23:19.595915 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c541dcc-a5af-40f0-b765-66ea04973ebe None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:23:19.596013 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c541dcc-a5af-40f0-b765-66ea04973ebe None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:23:19.597515 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 469/924] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:23:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:23:19.603616 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe4861c-8dd6-4531-a207-b7bbeefe94f1 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:23:19.604101 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe4861c-8dd6-4531-a207-b7bbeefe94f1 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:23:19.604204 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe4861c-8dd6-4531-a207-b7bbeefe94f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:23:19.621668 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-9fe4861c-8dd6-4531-a207-b7bbeefe94f1 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:23:19.622844 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-9fe4861c-8dd6-4531-a207-b7bbeefe94f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:23:19.691496 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 456/925] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:23:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 31 12:23:19.958487 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c541dcc-a5af-40f0-b765-66ea04973ebe None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:23:20.065149 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c541dcc-a5af-40f0-b765-66ea04973ebe None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:23:20.106546 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc38e982-2e15-42e4-8d36-27e393755849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:23:20.107202 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc38e982-2e15-42e4-8d36-27e393755849 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:23:20.107547 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc38e982-2e15-42e4-8d36-27e393755849 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:23:20.107818 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc38e982-2e15-42e4-8d36-27e393755849 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:23:20.114629 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:23:20.118791 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc38e982-2e15-42e4-8d36-27e393755849 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 31 12:23:20.124557 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-dc38e982-2e15-42e4-8d36-27e393755849 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 31 12:23:20.124557 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:23:20.124557 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:23:20.124557 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:23:20.124557 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:23:20.124557 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:23:20.124557 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:23:20.124557 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:23:20.124557 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:23:20.124557 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:23:20.124557 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:23:20.124557 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:23:20.124557 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 31 12:23:20.124557 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 31 12:23:20.124557 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 31 12:23:20.124557 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 31 12:23:20.125479 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 31 12:23:20.125479 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 31 12:23:20.125479 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 31 12:23:20.125479 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 31 12:23:20.125479 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:23:20.125479 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:23:20.125479 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:23:20.125479 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:23:20.125479 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:23:20.125479 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:23:20.125479 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:23:20.125479 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 31 12:23:20.125479 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:23:20.125479 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:23:20.125479 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:23:20.125479 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:23:20.126302 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:23:20.126302 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:23:20.126302 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:23:20.126302 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:23:20.126302 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 31 12:23:20.126302 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 31 12:23:20.126302 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 31 12:23:20.126302 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 31 12:23:20.126302 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 31 12:23:20.126302 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 31 12:23:20.126302 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 31 12:23:20.126302 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:23:20.131949 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 470/926] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Oct 31 12:23:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:23:20.133032 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe4861c-8dd6-4531-a207-b7bbeefe94f1 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:23:20.191077 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:23:20.241943 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe4861c-8dd6-4531-a207-b7bbeefe94f1 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:23:20.278196 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af2d8b7-52e0-4856-8b22-fd214ab4247a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:23:20.278796 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af2d8b7-52e0-4856-8b22-fd214ab4247a None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:23:20.279070 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af2d8b7-52e0-4856-8b22-fd214ab4247a None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:23:20.279368 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af2d8b7-52e0-4856-8b22-fd214ab4247a None admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:23:20.280015 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af2d8b7-52e0-4856-8b22-fd214ab4247a None admin] RBAC: Adding query filter params (name=service) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:23:20.280311 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af2d8b7-52e0-4856-8b22-fd214ab4247a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:23:20.281655 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af2d8b7-52e0-4856-8b22-fd214ab4247a None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:23:20.290555 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 457/927] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Oct 31 12:23:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:23:20.296493 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc38e982-2e15-42e4-8d36-27e393755849 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:23:20.408343 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc38e982-2e15-42e4-8d36-27e393755849 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:23:20.445112 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96634495-5da5-4836-aa43-55c8f0cf7b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:23:20.445744 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96634495-5da5-4836-aa43-55c8f0cf7b1a None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:23:20.446073 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96634495-5da5-4836-aa43-55c8f0cf7b1a None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:23:20.446370 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96634495-5da5-4836-aa43-55c8f0cf7b1a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:23:20.457287 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96634495-5da5-4836-aa43-55c8f0cf7b1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:23:20.458911 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96634495-5da5-4836-aa43-55c8f0cf7b1a None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:23:20.474462 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-96634495-5da5-4836-aa43-55c8f0cf7b1a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:23:20.474462 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:23:20.474462 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:23:20.474462 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:23:20.474462 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:23:20.474462 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:23:20.474462 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:23:20.474462 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:23:20.474462 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:23:20.474462 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:23:20.474462 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:23:20.474462 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:23:20.474462 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 31 12:23:20.474462 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 31 12:23:20.474462 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 31 12:23:20.474462 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 31 12:23:20.475514 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:23:20.475514 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:23:20.475514 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:23:20.475514 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:23:20.475514 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:23:20.475514 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:23:20.475514 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:23:20.475514 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 31 12:23:20.475514 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:23:20.475514 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:23:20.475514 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:23:20.475514 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:23:20.475514 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:23:20.475514 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:23:20.475514 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:23:20.475514 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:23:20.475514 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 31 12:23:20.476424 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 31 12:23:20.476424 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:23:20.476424 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:23:20.479503 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 471/928] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Oct 31 12:23:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:23:20.481402 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af2d8b7-52e0-4856-8b22-fd214ab4247a None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:23:20.588887 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af2d8b7-52e0-4856-8b22-fd214ab4247a None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:23:20.605078 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:23:20.624521 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567d0759-094f-408e-91d2-e0d21a400ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:23:20.625117 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567d0759-094f-408e-91d2-e0d21a400ee5 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:23:20.625390 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567d0759-094f-408e-91d2-e0d21a400ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:23:20.625652 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567d0759-094f-408e-91d2-e0d21a400ee5 None admin] PATH_INFO: `/v3/domains` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:23:20.626307 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567d0759-094f-408e-91d2-e0d21a400ee5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:23:20.626600 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567d0759-094f-408e-91d2-e0d21a400ee5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:23:20.627974 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567d0759-094f-408e-91d2-e0d21a400ee5 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:23:20.646616 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 458/929] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Oct 31 12:23:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:23:20.648037 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96634495-5da5-4836-aa43-55c8f0cf7b1a None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:23:20.703696 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:23:20.761552 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96634495-5da5-4836-aa43-55c8f0cf7b1a None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:23:20.796935 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d1cdbe-db57-48d1-a2bc-def3002d5bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:23:20.797818 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d1cdbe-db57-48d1-a2bc-def3002d5bc7 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:23:20.798085 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d1cdbe-db57-48d1-a2bc-def3002d5bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:23:20.798348 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d1cdbe-db57-48d1-a2bc-def3002d5bc7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:23:20.806963 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d1cdbe-db57-48d1-a2bc-def3002d5bc7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:23:20.808412 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d1cdbe-db57-48d1-a2bc-def3002d5bc7 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:23:20.819457 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-30d1cdbe-db57-48d1-a2bc-def3002d5bc7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 31 12:23:20.819457 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:23:20.819457 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:23:20.819457 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:23:20.819457 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:23:20.819457 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:23:20.819457 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:23:20.819457 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:23:20.819457 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:23:20.819457 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:23:20.819457 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:23:20.819457 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:23:20.819457 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 31 12:23:20.819457 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 31 12:23:20.819457 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 31 12:23:20.819457 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 31 12:23:20.820644 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:23:20.820644 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:23:20.820644 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 31 12:23:20.820644 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 31 12:23:20.820644 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 31 12:23:20.820644 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 31 12:23:20.820644 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:23:20.820644 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:23:20.820644 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:23:20.820644 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:23:20.820644 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:23:20.820644 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 31 12:23:20.820644 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:23:20.820644 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:23:20.820644 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:23:20.820644 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:23:20.820644 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:23:20.820644 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:23:20.821807 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:23:20.821807 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:23:20.821807 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 31 12:23:20.821807 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 31 12:23:20.821807 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 31 12:23:20.821807 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 31 12:23:20.821807 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 31 12:23:20.821807 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 31 12:23:20.821807 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 31 12:23:20.821807 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:23:20.821807 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 472/930] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Oct 31 12:23:20 2022] GET /identity/v3/users/placement => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 31 12:23:20.826846 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567d0759-094f-408e-91d2-e0d21a400ee5 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:23:20.938473 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567d0759-094f-408e-91d2-e0d21a400ee5 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:23:20.977793 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66a0a5e-b00d-4fb0-a54c-479a59783cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:23:20.978545 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66a0a5e-b00d-4fb0-a54c-479a59783cc8 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:23:20.978856 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66a0a5e-b00d-4fb0-a54c-479a59783cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:23:20.979183 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66a0a5e-b00d-4fb0-a54c-479a59783cc8 None admin] PATH_INFO: `/v3/users` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:23:20.979951 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66a0a5e-b00d-4fb0-a54c-479a59783cc8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:23:20.980275 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66a0a5e-b00d-4fb0-a54c-479a59783cc8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:23:20.982061 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66a0a5e-b00d-4fb0-a54c-479a59783cc8 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:23:21.001483 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 459/931] 10.222.0.15 () {56 vars in 1143 bytes} [Mon Oct 31 12:23:20 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:23:21.008348 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d1cdbe-db57-48d1-a2bc-def3002d5bc7 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:23:21.111698 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d1cdbe-db57-48d1-a2bc-def3002d5bc7 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:23:21.122711 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:23:21.147520 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1d3407-23ef-40ff-a113-4cb52098c5e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:23:21.148127 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d3407-23ef-40ff-a113-4cb52098c5e6 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:23:21.148400 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d3407-23ef-40ff-a113-4cb52098c5e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:23:21.148665 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d3407-23ef-40ff-a113-4cb52098c5e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:23:21.225130 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1d3407-23ef-40ff-a113-4cb52098c5e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:23:21.226496 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1d3407-23ef-40ff-a113-4cb52098c5e6 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:23:21.339113 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 473/932] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Oct 31 12:23:21 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 31 12:23:21.346289 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66a0a5e-b00d-4fb0-a54c-479a59783cc8 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:23:21.386137 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:23:21.468708 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66a0a5e-b00d-4fb0-a54c-479a59783cc8 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:23:21.505291 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4067ee5-6b90-4479-9cb7-8583c4a20636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:23:21.505291 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4067ee5-6b90-4479-9cb7-8583c4a20636 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:23:21.505291 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4067ee5-6b90-4479-9cb7-8583c4a20636 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:23:21.505291 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4067ee5-6b90-4479-9cb7-8583c4a20636 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:23:21.515250 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4067ee5-6b90-4479-9cb7-8583c4a20636 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:23:21.516810 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4067ee5-6b90-4479-9cb7-8583c4a20636 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:23:21.533188 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-b4067ee5-6b90-4479-9cb7-8583c4a20636 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:23:21.533188 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:23:21.533188 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:23:21.533188 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:23:21.533188 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:23:21.533188 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:23:21.533188 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:23:21.533188 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:23:21.533188 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:23:21.533188 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:23:21.533188 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:23:21.533188 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:23:21.533188 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 31 12:23:21.533188 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 31 12:23:21.533188 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 31 12:23:21.533188 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 31 12:23:21.534316 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:23:21.534316 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:23:21.534316 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:23:21.534316 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:23:21.534316 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:23:21.534316 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:23:21.534316 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:23:21.534316 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:23:21.534316 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:23:21.534316 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:23:21.534316 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:23:21.534316 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:23:21.534316 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:23:21.534316 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:23:21.534316 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:23:21.534316 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:23:21.534316 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 31 12:23:21.535377 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 31 12:23:21.535377 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:23:21.535377 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:23:21.535377 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 460/933] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Oct 31 12:23:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:23:21.540710 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1d3407-23ef-40ff-a113-4cb52098c5e6 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:23:21.553061 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:23:21.645232 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1d3407-23ef-40ff-a113-4cb52098c5e6 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:23:21.683400 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4daf23e-3d2d-4ea1-8481-153dcc644e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:23:21.684064 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4daf23e-3d2d-4ea1-8481-153dcc644e7b None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:23:21.684288 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4daf23e-3d2d-4ea1-8481-153dcc644e7b None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:23:21.684718 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4daf23e-3d2d-4ea1-8481-153dcc644e7b None admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:23:21.685429 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4daf23e-3d2d-4ea1-8481-153dcc644e7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:23:21.685825 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4daf23e-3d2d-4ea1-8481-153dcc644e7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:23:21.687753 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4daf23e-3d2d-4ea1-8481-153dcc644e7b None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:23:21.702757 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 474/934] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Oct 31 12:23:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:23:21.709847 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4067ee5-6b90-4479-9cb7-8583c4a20636 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:23:21.804904 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:23:21.821449 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4067ee5-6b90-4479-9cb7-8583c4a20636 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:23:21.859471 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cab4197-e0d8-4f97-a096-3e43c519e155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:23:21.860210 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cab4197-e0d8-4f97-a096-3e43c519e155 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:23:21.860549 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cab4197-e0d8-4f97-a096-3e43c519e155 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:23:21.860827 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cab4197-e0d8-4f97-a096-3e43c519e155 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:23:21.868760 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cab4197-e0d8-4f97-a096-3e43c519e155 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:23:21.870192 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cab4197-e0d8-4f97-a096-3e43c519e155 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:23:21.881545 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-4cab4197-e0d8-4f97-a096-3e43c519e155 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 31 12:23:21.881545 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:23:21.881545 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:23:21.881545 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:23:21.881545 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:23:21.881545 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:23:21.881545 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:23:21.881545 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:23:21.881545 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:23:21.881545 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:23:21.881545 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:23:21.881545 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:23:21.881545 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 31 12:23:21.881545 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 31 12:23:21.881545 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 31 12:23:21.882646 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 31 12:23:21.882646 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:23:21.882646 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:23:21.882646 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:23:21.882646 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:23:21.882646 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:23:21.882646 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:23:21.882646 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:23:21.882646 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:23:21.882646 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:23:21.882646 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:23:21.882646 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:23:21.882646 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:23:21.882646 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:23:21.882646 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:23:21.882646 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:23:21.882646 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:23:21.883566 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 31 12:23:21.883566 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 31 12:23:21.883566 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 31 12:23:21.883566 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 31 12:23:21.883566 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 31 12:23:21.883566 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 31 12:23:21.883566 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 31 12:23:21.883566 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:23:21.883566 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 461/935] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Oct 31 12:23:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:23:21.889283 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4daf23e-3d2d-4ea1-8481-153dcc644e7b None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:23:21.966578 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:23:22.004617 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4daf23e-3d2d-4ea1-8481-153dcc644e7b None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:23:22.051004 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c40cba8-a1c2-48b8-82f3-2370623f4bd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:23:22.051681 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c40cba8-a1c2-48b8-82f3-2370623f4bd7 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:23:22.051957 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c40cba8-a1c2-48b8-82f3-2370623f4bd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:23:22.052222 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c40cba8-a1c2-48b8-82f3-2370623f4bd7 None admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:23:22.053563 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c40cba8-a1c2-48b8-82f3-2370623f4bd7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:23:22.053864 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c40cba8-a1c2-48b8-82f3-2370623f4bd7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:23:22.055483 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c40cba8-a1c2-48b8-82f3-2370623f4bd7 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:23:22.073794 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 475/936] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Oct 31 12:23:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 31 12:23:22.075085 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cab4197-e0d8-4f97-a096-3e43c519e155 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:23:22.183605 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cab4197-e0d8-4f97-a096-3e43c519e155 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:23:22.219326 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6811ce2-bc17-4a59-8778-a42a03ae9037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:23:22.219931 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6811ce2-bc17-4a59-8778-a42a03ae9037 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:23:22.220211 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6811ce2-bc17-4a59-8778-a42a03ae9037 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:23:22.220483 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6811ce2-bc17-4a59-8778-a42a03ae9037 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:23:22.221230 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6811ce2-bc17-4a59-8778-a42a03ae9037 None admin] RBAC: Adding query filter params (role.id=9674a2c5acc74dfeb93cfd67dfd890dd, scope.project.id=2c32444cef584bce966b38203ce128b2, user.id=9e7777495d7a406c8c50b1ad52e47263) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:23:22.221569 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6811ce2-bc17-4a59-8778-a42a03ae9037 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:23:22.223267 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6811ce2-bc17-4a59-8778-a42a03ae9037 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:23:22.229988 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 462/937] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Oct 31 12:23:22 2022] GET /identity/v3/role_assignments?user.id=9e7777495d7a406c8c50b1ad52e47263&scope.project.id=2c32444cef584bce966b38203ce128b2&role.id=9674a2c5acc74dfeb93cfd67dfd890dd => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:23:24.413937 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d199fd5-0b16-4ca5-8540-c56bd1bc0056 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:23:24.414495 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d199fd5-0b16-4ca5-8540-c56bd1bc0056 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:23:24.414495 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d199fd5-0b16-4ca5-8540-c56bd1bc0056 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:23:24.416322 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 476/938] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:23:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:23:24.421301 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174fae53-9e8b-4749-9cc1-5a0cb86f3e79 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:23:24.421817 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174fae53-9e8b-4749-9cc1-5a0cb86f3e79 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:23:24.421817 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174fae53-9e8b-4749-9cc1-5a0cb86f3e79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:23:24.442341 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-174fae53-9e8b-4749-9cc1-5a0cb86f3e79 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:23:24.444314 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-174fae53-9e8b-4749-9cc1-5a0cb86f3e79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:23:24.447662 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:23:24.517910 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 463/939] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:23:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 31 12:23:24.833318 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d199fd5-0b16-4ca5-8540-c56bd1bc0056 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:23:24.942074 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d199fd5-0b16-4ca5-8540-c56bd1bc0056 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:23:24.970532 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:23:24.977165 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886c23f2-98fe-403d-955a-393cbd7c6009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:23:24.977790 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886c23f2-98fe-403d-955a-393cbd7c6009 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:23:24.978065 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886c23f2-98fe-403d-955a-393cbd7c6009 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:23:24.978398 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886c23f2-98fe-403d-955a-393cbd7c6009 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:23:24.991962 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886c23f2-98fe-403d-955a-393cbd7c6009 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 31 12:23:24.997188 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-886c23f2-98fe-403d-955a-393cbd7c6009 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 31 12:23:24.997188 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:23:24.997188 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:23:24.997188 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:23:24.997188 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:23:24.997188 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:23:24.997188 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:23:24.997188 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:23:24.997188 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:23:24.997188 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:23:24.997188 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:23:24.997188 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:23:24.997188 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 31 12:23:24.997188 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 31 12:23:24.997188 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 31 12:23:24.997188 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 31 12:23:24.998268 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 31 12:23:24.998268 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 31 12:23:24.998268 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 31 12:23:24.998268 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 31 12:23:24.998268 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:23:24.998268 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:23:24.998268 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:23:24.998268 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:23:24.998268 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:23:24.998268 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:23:24.998268 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:23:24.998268 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 31 12:23:24.998268 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:23:24.998268 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:23:24.998268 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:23:24.998268 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:23:24.999114 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:23:24.999114 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:23:24.999114 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:23:24.999114 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:23:24.999114 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 31 12:23:24.999114 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 31 12:23:24.999114 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 31 12:23:24.999114 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 31 12:23:24.999114 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 31 12:23:24.999114 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 31 12:23:24.999114 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 31 12:23:24.999114 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:23:24.999114 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 477/940] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Oct 31 12:23:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:23:25.004461 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174fae53-9e8b-4749-9cc1-5a0cb86f3e79 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:23:25.122568 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174fae53-9e8b-4749-9cc1-5a0cb86f3e79 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:23:25.159467 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37457ab-7285-4111-bc2b-08d26c1b8b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:23:25.160067 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37457ab-7285-4111-bc2b-08d26c1b8b5f None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:23:25.160388 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37457ab-7285-4111-bc2b-08d26c1b8b5f None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:23:25.160644 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37457ab-7285-4111-bc2b-08d26c1b8b5f None admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:23:25.161356 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37457ab-7285-4111-bc2b-08d26c1b8b5f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:23:25.161758 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37457ab-7285-4111-bc2b-08d26c1b8b5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:23:25.163334 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37457ab-7285-4111-bc2b-08d26c1b8b5f None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:23:25.172467 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 464/941] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Oct 31 12:23:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:23:25.178978 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886c23f2-98fe-403d-955a-393cbd7c6009 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:23:25.288934 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886c23f2-98fe-403d-955a-393cbd7c6009 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:23:25.327227 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e88d214-7605-4374-b174-d1e5c50268c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:23:25.327979 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e88d214-7605-4374-b174-d1e5c50268c1 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:23:25.328377 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e88d214-7605-4374-b174-d1e5c50268c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:23:25.328648 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e88d214-7605-4374-b174-d1e5c50268c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:23:25.340939 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e88d214-7605-4374-b174-d1e5c50268c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:23:25.342514 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e88d214-7605-4374-b174-d1e5c50268c1 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:23:25.358645 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-9e88d214-7605-4374-b174-d1e5c50268c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:23:25.358645 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:23:25.358645 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:23:25.358645 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:23:25.358645 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:23:25.358645 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:23:25.358645 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:23:25.358645 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:23:25.358645 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:23:25.358645 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:23:25.358645 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:23:25.358645 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:23:25.358645 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 31 12:23:25.358645 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 31 12:23:25.358645 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 31 12:23:25.358645 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 31 12:23:25.359722 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:23:25.359722 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:23:25.359722 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:23:25.359722 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:23:25.359722 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:23:25.359722 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:23:25.359722 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:23:25.359722 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 31 12:23:25.359722 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:23:25.359722 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:23:25.359722 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:23:25.359722 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:23:25.359722 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:23:25.359722 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:23:25.359722 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:23:25.359722 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:23:25.359722 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 31 12:23:25.360603 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 31 12:23:25.360603 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:23:25.360603 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:23:25.360603 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 478/942] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Oct 31 12:23:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 31 12:23:25.366244 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37457ab-7285-4111-bc2b-08d26c1b8b5f None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:23:25.369499 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:23:25.478374 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37457ab-7285-4111-bc2b-08d26c1b8b5f None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:23:25.517318 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df9f416-1f8a-42df-bcef-0735d0e53b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:23:25.517920 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df9f416-1f8a-42df-bcef-0735d0e53b2e None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:23:25.518199 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df9f416-1f8a-42df-bcef-0735d0e53b2e None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:23:25.518469 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df9f416-1f8a-42df-bcef-0735d0e53b2e None admin] PATH_INFO: `/v3/domains` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:23:25.519386 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df9f416-1f8a-42df-bcef-0735d0e53b2e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:23:25.519687 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df9f416-1f8a-42df-bcef-0735d0e53b2e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:23:25.521053 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df9f416-1f8a-42df-bcef-0735d0e53b2e None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:23:25.532729 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 465/943] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Oct 31 12:23:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:23:25.539080 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e88d214-7605-4374-b174-d1e5c50268c1 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:23:25.583375 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:23:25.648127 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e88d214-7605-4374-b174-d1e5c50268c1 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:23:25.682607 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c70fad3-5642-4ee0-b3b7-c882e51557e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:23:25.683266 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c70fad3-5642-4ee0-b3b7-c882e51557e3 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:23:25.683547 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c70fad3-5642-4ee0-b3b7-c882e51557e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:23:25.683818 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c70fad3-5642-4ee0-b3b7-c882e51557e3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:23:25.691105 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c70fad3-5642-4ee0-b3b7-c882e51557e3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:23:25.692543 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c70fad3-5642-4ee0-b3b7-c882e51557e3 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:23:25.704346 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-0c70fad3-5642-4ee0-b3b7-c882e51557e3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 31 12:23:25.704346 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:23:25.704346 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:23:25.704346 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:23:25.704346 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:23:25.704346 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:23:25.704346 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:23:25.704346 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:23:25.704346 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:23:25.704346 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:23:25.704346 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:23:25.704346 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:23:25.704346 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 31 12:23:25.704346 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 31 12:23:25.704346 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 31 12:23:25.704346 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 31 12:23:25.705429 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:23:25.705429 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:23:25.705429 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 31 12:23:25.705429 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 31 12:23:25.705429 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 31 12:23:25.705429 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 31 12:23:25.705429 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:23:25.705429 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:23:25.705429 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:23:25.705429 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:23:25.705429 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:23:25.705429 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 31 12:23:25.705429 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:23:25.705429 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:23:25.705429 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:23:25.705429 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:23:25.705429 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:23:25.705429 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:23:25.706394 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:23:25.706394 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:23:25.706394 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 31 12:23:25.706394 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 31 12:23:25.706394 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 31 12:23:25.706394 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 31 12:23:25.706394 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 31 12:23:25.706394 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 31 12:23:25.706394 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 31 12:23:25.706394 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:23:25.706394 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 479/944] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Oct 31 12:23:25 2022] GET /identity/v3/users/placement => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:23:25.711953 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df9f416-1f8a-42df-bcef-0735d0e53b2e None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:23:25.787582 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:23:25.830753 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df9f416-1f8a-42df-bcef-0735d0e53b2e None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:23:25.869819 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3993aa49-bb70-45d2-a495-9b3a90c15262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:23:25.870567 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3993aa49-bb70-45d2-a495-9b3a90c15262 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:23:25.870922 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3993aa49-bb70-45d2-a495-9b3a90c15262 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:23:25.871290 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3993aa49-bb70-45d2-a495-9b3a90c15262 None admin] PATH_INFO: `/v3/users` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:23:25.872014 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3993aa49-bb70-45d2-a495-9b3a90c15262 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:23:25.872320 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3993aa49-bb70-45d2-a495-9b3a90c15262 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:23:25.873732 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3993aa49-bb70-45d2-a495-9b3a90c15262 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:23:25.895798 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 466/945] 10.222.0.15 () {56 vars in 1143 bytes} [Mon Oct 31 12:23:25 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:23:25.902324 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c70fad3-5642-4ee0-b3b7-c882e51557e3 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:23:26.010877 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c70fad3-5642-4ee0-b3b7-c882e51557e3 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:23:26.012450 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:23:26.048829 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c351adbb-2eb9-4d91-8d25-76881ea58743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:23:26.049297 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c351adbb-2eb9-4d91-8d25-76881ea58743 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:23:26.049570 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c351adbb-2eb9-4d91-8d25-76881ea58743 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:23:26.049842 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c351adbb-2eb9-4d91-8d25-76881ea58743 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:23:26.060457 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c351adbb-2eb9-4d91-8d25-76881ea58743 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:23:26.061882 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c351adbb-2eb9-4d91-8d25-76881ea58743 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:23:26.076321 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-c351adbb-2eb9-4d91-8d25-76881ea58743 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:23:26.076321 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:23:26.076321 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:23:26.076321 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:23:26.076321 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:23:26.076321 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:23:26.076321 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:23:26.076321 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:23:26.076321 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:23:26.076321 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:23:26.076321 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:23:26.076321 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:23:26.076321 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 31 12:23:26.076321 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 31 12:23:26.076321 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 31 12:23:26.076321 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 31 12:23:26.077464 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:23:26.077464 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:23:26.077464 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:23:26.077464 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:23:26.077464 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:23:26.077464 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:23:26.077464 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:23:26.077464 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 31 12:23:26.077464 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:23:26.077464 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:23:26.077464 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:23:26.077464 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:23:26.077464 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:23:26.077464 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:23:26.077464 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:23:26.077464 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:23:26.077464 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 31 12:23:26.078552 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 31 12:23:26.078552 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:23:26.078552 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:23:26.078552 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 480/946] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Oct 31 12:23:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:23:26.083464 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3993aa49-bb70-45d2-a495-9b3a90c15262 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:23:26.192671 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3993aa49-bb70-45d2-a495-9b3a90c15262 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:23:26.216527 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:23:26.232342 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afe0035-8d73-4114-876b-b2f0f783e9b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:23:26.232948 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afe0035-8d73-4114-876b-b2f0f783e9b7 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:23:26.233224 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afe0035-8d73-4114-876b-b2f0f783e9b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:23:26.233492 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afe0035-8d73-4114-876b-b2f0f783e9b7 None admin] PATH_INFO: `/v3/domains` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:23:26.234129 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afe0035-8d73-4114-876b-b2f0f783e9b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:23:26.235780 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afe0035-8d73-4114-876b-b2f0f783e9b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:23:26.237475 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afe0035-8d73-4114-876b-b2f0f783e9b7 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:23:26.250086 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 467/947] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Oct 31 12:23:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:23:26.256476 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c351adbb-2eb9-4d91-8d25-76881ea58743 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:23:26.362078 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c351adbb-2eb9-4d91-8d25-76881ea58743 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:23:26.406105 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3438f89-8ecf-450b-aa2c-f5b9da2b2109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:23:26.406830 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3438f89-8ecf-450b-aa2c-f5b9da2b2109 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:23:26.407149 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3438f89-8ecf-450b-aa2c-f5b9da2b2109 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:23:26.407481 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3438f89-8ecf-450b-aa2c-f5b9da2b2109 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:23:26.415787 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3438f89-8ecf-450b-aa2c-f5b9da2b2109 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:23:26.417525 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3438f89-8ecf-450b-aa2c-f5b9da2b2109 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:23:26.430579 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-f3438f89-8ecf-450b-aa2c-f5b9da2b2109 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 31 12:23:26.430579 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:23:26.430579 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:23:26.430579 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:23:26.430579 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:23:26.430579 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:23:26.430579 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:23:26.430579 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:23:26.430579 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:23:26.430579 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:23:26.430579 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:23:26.430579 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:23:26.430579 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 31 12:23:26.430579 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 31 12:23:26.430579 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 31 12:23:26.431727 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 31 12:23:26.431727 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:23:26.431727 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:23:26.431727 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:23:26.431727 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:23:26.431727 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:23:26.431727 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:23:26.431727 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:23:26.431727 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 31 12:23:26.431727 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:23:26.431727 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:23:26.431727 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:23:26.431727 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:23:26.431727 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:23:26.431727 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:23:26.431727 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:23:26.431727 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:23:26.432618 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 31 12:23:26.432618 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 31 12:23:26.432618 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 31 12:23:26.432618 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 31 12:23:26.432618 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 31 12:23:26.432618 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 31 12:23:26.432618 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 31 12:23:26.432618 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:23:26.432618 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 481/948] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Oct 31 12:23:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:23:26.437724 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afe0035-8d73-4114-876b-b2f0f783e9b7 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:23:26.545105 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afe0035-8d73-4114-876b-b2f0f783e9b7 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:23:26.583723 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e44706-79de-440a-9517-4f8040edac9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:23:26.584507 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e44706-79de-440a-9517-4f8040edac9a None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:23:26.584821 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e44706-79de-440a-9517-4f8040edac9a None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:23:26.585089 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e44706-79de-440a-9517-4f8040edac9a None admin] PATH_INFO: `/v3/projects` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:23:26.585900 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e44706-79de-440a-9517-4f8040edac9a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:23:26.586202 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e44706-79de-440a-9517-4f8040edac9a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:23:26.587644 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e44706-79de-440a-9517-4f8040edac9a None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:23:26.605412 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 468/949] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Oct 31 12:23:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:23:26.607092 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3438f89-8ecf-450b-aa2c-f5b9da2b2109 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:23:26.628891 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:23:26.733315 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3438f89-8ecf-450b-aa2c-f5b9da2b2109 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:23:26.773218 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97987f9a-1d0c-4531-a240-f394f2137754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:23:26.773926 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97987f9a-1d0c-4531-a240-f394f2137754 None admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:23:26.774208 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97987f9a-1d0c-4531-a240-f394f2137754 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:23:26.774534 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97987f9a-1d0c-4531-a240-f394f2137754 None admin] PATH_INFO: `/v3/projects/2c32444cef584bce966b38203ce128b2/users/9e7777495d7a406c8c50b1ad52e47263/roles/176c07b4fe3b4537bdfeb4be21b379c5` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:23:26.776534 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97987f9a-1d0c-4531-a240-f394f2137754 None admin] RBAC: Authorizing `identity:create_grant(project_id=2c32444cef584bce966b38203ce128b2, user_id=9e7777495d7a406c8c50b1ad52e47263, role_id=176c07b4fe3b4537bdfeb4be21b379c5)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:23:26.777990 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97987f9a-1d0c-4531-a240-f394f2137754 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:23:26.790882 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 482/950] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Oct 31 12:23:26 2022] PUT /identity/v3/projects/2c32444cef584bce966b38203ce128b2/users/9e7777495d7a406c8c50b1ad52e47263/roles/176c07b4fe3b4537bdfeb4be21b379c5 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:23:29.048562 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e2e09f-7a6b-41d1-94f1-7c0bd7b3daa6 None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:23:29.049100 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e2e09f-7a6b-41d1-94f1-7c0bd7b3daa6 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:23:29.049100 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e2e09f-7a6b-41d1-94f1-7c0bd7b3daa6 None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:23:29.050453 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 469/951] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:23:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:23:29.056718 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188529c1-43eb-443d-b089-793a3e44d884 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:23:29.057173 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188529c1-43eb-443d-b089-793a3e44d884 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:23:29.057250 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188529c1-43eb-443d-b089-793a3e44d884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:23:29.075850 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-188529c1-43eb-443d-b089-793a3e44d884 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:23:29.077068 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-188529c1-43eb-443d-b089-793a3e44d884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:23:29.149669 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 483/952] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:23:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 31 12:23:29.421459 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e2e09f-7a6b-41d1-94f1-7c0bd7b3daa6 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:23:29.469587 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:23:29.527708 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e2e09f-7a6b-41d1-94f1-7c0bd7b3daa6 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:23:29.566071 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abafb897-4316-4b3b-83fa-a1c17466e911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:23:29.566689 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abafb897-4316-4b3b-83fa-a1c17466e911 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:23:29.566963 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abafb897-4316-4b3b-83fa-a1c17466e911 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:23:29.567257 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abafb897-4316-4b3b-83fa-a1c17466e911 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:23:29.577696 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abafb897-4316-4b3b-83fa-a1c17466e911 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 31 12:23:29.582631 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-abafb897-4316-4b3b-83fa-a1c17466e911 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 31 12:23:29.582631 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:23:29.582631 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:23:29.582631 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:23:29.582631 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:23:29.582631 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:23:29.582631 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:23:29.582631 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:23:29.582631 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:23:29.582631 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:23:29.582631 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:23:29.582631 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:23:29.582631 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 31 12:23:29.582631 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 31 12:23:29.582631 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 31 12:23:29.582631 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 31 12:23:29.584193 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 31 12:23:29.584193 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 31 12:23:29.584193 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 31 12:23:29.584193 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 31 12:23:29.584193 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:23:29.584193 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:23:29.584193 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:23:29.584193 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:23:29.584193 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:23:29.584193 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:23:29.584193 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:23:29.584193 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:23:29.584193 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:23:29.584193 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:23:29.584193 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:23:29.584193 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:23:29.585031 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:23:29.585031 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:23:29.585031 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:23:29.585031 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:23:29.585031 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 31 12:23:29.585031 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 31 12:23:29.585031 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 31 12:23:29.585031 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 31 12:23:29.585031 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 31 12:23:29.585031 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 31 12:23:29.585031 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 31 12:23:29.585031 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:23:29.585031 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 470/953] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Oct 31 12:23:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:23:29.591028 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188529c1-43eb-443d-b089-793a3e44d884 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:23:29.594175 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:23:29.706374 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188529c1-43eb-443d-b089-793a3e44d884 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:23:29.748800 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b4fb7e-d727-4c3c-96f8-2b004199b3bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:23:29.749498 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b4fb7e-d727-4c3c-96f8-2b004199b3bf None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:23:29.749980 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b4fb7e-d727-4c3c-96f8-2b004199b3bf None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:23:29.750322 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b4fb7e-d727-4c3c-96f8-2b004199b3bf None admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:23:29.751266 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b4fb7e-d727-4c3c-96f8-2b004199b3bf None admin] RBAC: Adding query filter params (name=admin) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:23:29.751634 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b4fb7e-d727-4c3c-96f8-2b004199b3bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:23:29.753181 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b4fb7e-d727-4c3c-96f8-2b004199b3bf None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:23:29.767199 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 484/954] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Oct 31 12:23:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:23:29.771705 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abafb897-4316-4b3b-83fa-a1c17466e911 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:23:29.885555 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:23:29.888861 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abafb897-4316-4b3b-83fa-a1c17466e911 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:23:29.924508 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47963630-0f56-4845-81bc-9e28dfbb13fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:23:29.925189 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47963630-0f56-4845-81bc-9e28dfbb13fe None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:23:29.925574 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47963630-0f56-4845-81bc-9e28dfbb13fe None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:23:29.925888 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47963630-0f56-4845-81bc-9e28dfbb13fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:23:29.937048 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47963630-0f56-4845-81bc-9e28dfbb13fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:23:29.938489 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47963630-0f56-4845-81bc-9e28dfbb13fe None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:23:29.956323 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-47963630-0f56-4845-81bc-9e28dfbb13fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:23:29.956323 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:23:29.956323 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:23:29.956323 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:23:29.956323 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:23:29.956323 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:23:29.956323 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:23:29.956323 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:23:29.956323 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:23:29.956323 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:23:29.956323 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:23:29.956323 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:23:29.956323 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 31 12:23:29.956323 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 31 12:23:29.956323 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 31 12:23:29.956323 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 31 12:23:29.957618 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:23:29.957618 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:23:29.957618 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:23:29.957618 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:23:29.957618 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:23:29.957618 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:23:29.957618 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:23:29.957618 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:23:29.957618 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:23:29.957618 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:23:29.957618 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:23:29.957618 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:23:29.957618 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:23:29.957618 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:23:29.957618 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:23:29.957618 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:23:29.957618 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 31 12:23:29.958769 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 31 12:23:29.958769 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:23:29.958769 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:23:29.958769 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 471/955] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Oct 31 12:23:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:23:29.963647 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b4fb7e-d727-4c3c-96f8-2b004199b3bf None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:23:30.032657 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:23:30.070265 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b4fb7e-d727-4c3c-96f8-2b004199b3bf None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:23:30.106971 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5093d58c-f5cd-4e5b-807e-88913cfda2d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:23:30.107613 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5093d58c-f5cd-4e5b-807e-88913cfda2d2 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:23:30.107934 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5093d58c-f5cd-4e5b-807e-88913cfda2d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:23:30.108259 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5093d58c-f5cd-4e5b-807e-88913cfda2d2 None admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:23:30.109041 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5093d58c-f5cd-4e5b-807e-88913cfda2d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:23:30.109388 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5093d58c-f5cd-4e5b-807e-88913cfda2d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:23:30.110845 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5093d58c-f5cd-4e5b-807e-88913cfda2d2 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:23:30.124522 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 485/956] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Oct 31 12:23:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:23:30.130193 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47963630-0f56-4845-81bc-9e28dfbb13fe None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:23:30.248830 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47963630-0f56-4845-81bc-9e28dfbb13fe None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:23:30.282657 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1213680e-b65b-4797-a2a5-ac7373563362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:23:30.283311 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1213680e-b65b-4797-a2a5-ac7373563362 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:23:30.283590 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1213680e-b65b-4797-a2a5-ac7373563362 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:23:30.283858 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1213680e-b65b-4797-a2a5-ac7373563362 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:23:30.292521 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1213680e-b65b-4797-a2a5-ac7373563362 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:23:30.294385 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1213680e-b65b-4797-a2a5-ac7373563362 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:23:30.307651 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-1213680e-b65b-4797-a2a5-ac7373563362 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 31 12:23:30.307651 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:23:30.307651 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:23:30.307651 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:23:30.307651 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:23:30.307651 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:23:30.307651 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:23:30.307651 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:23:30.307651 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:23:30.307651 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:23:30.307651 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:23:30.307651 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:23:30.307651 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 31 12:23:30.307651 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 31 12:23:30.307651 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 31 12:23:30.307651 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 31 12:23:30.308794 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:23:30.308794 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:23:30.308794 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 31 12:23:30.308794 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 31 12:23:30.308794 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 31 12:23:30.308794 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 31 12:23:30.308794 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:23:30.308794 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:23:30.308794 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:23:30.308794 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:23:30.308794 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:23:30.308794 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:23:30.308794 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:23:30.308794 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:23:30.308794 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:23:30.308794 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:23:30.308794 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:23:30.308794 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:23:30.309919 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:23:30.309919 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:23:30.309919 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 31 12:23:30.309919 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 31 12:23:30.309919 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 31 12:23:30.309919 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 31 12:23:30.309919 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 31 12:23:30.309919 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 31 12:23:30.309919 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 31 12:23:30.309919 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:23:30.309919 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 472/957] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Oct 31 12:23:30 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:23:30.315182 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5093d58c-f5cd-4e5b-807e-88913cfda2d2 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:23:30.421798 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5093d58c-f5cd-4e5b-807e-88913cfda2d2 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:23:30.453085 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:23:30.463247 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13861d96-c039-4930-9e7b-5fb27c50f286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:23:30.463904 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13861d96-c039-4930-9e7b-5fb27c50f286 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:23:30.464191 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13861d96-c039-4930-9e7b-5fb27c50f286 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:23:30.464464 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13861d96-c039-4930-9e7b-5fb27c50f286 None admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:23:30.465480 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13861d96-c039-4930-9e7b-5fb27c50f286 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:23:30.465811 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13861d96-c039-4930-9e7b-5fb27c50f286 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:23:30.467496 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13861d96-c039-4930-9e7b-5fb27c50f286 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:23:30.485424 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 486/958] 10.222.0.15 () {56 vars in 1143 bytes} [Mon Oct 31 12:23:30 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:23:30.492143 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1213680e-b65b-4797-a2a5-ac7373563362 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:23:30.509559 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:23:30.601715 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1213680e-b65b-4797-a2a5-ac7373563362 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:23:30.635923 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f55246-7704-4490-904a-30e4df140b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:23:30.636538 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f55246-7704-4490-904a-30e4df140b65 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:23:30.636795 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f55246-7704-4490-904a-30e4df140b65 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:23:30.637060 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f55246-7704-4490-904a-30e4df140b65 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:23:30.726149 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f55246-7704-4490-904a-30e4df140b65 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:23:30.727778 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f55246-7704-4490-904a-30e4df140b65 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:23:30.744829 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:23:30.838898 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 473/959] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Oct 31 12:23:30 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 31 12:23:30.845011 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13861d96-c039-4930-9e7b-5fb27c50f286 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:23:30.955962 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13861d96-c039-4930-9e7b-5fb27c50f286 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:23:30.994249 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f16c8d-13a3-4625-9a90-c4744a5014cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:23:30.994886 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f16c8d-13a3-4625-9a90-c4744a5014cd None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:23:30.995202 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f16c8d-13a3-4625-9a90-c4744a5014cd None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:23:30.995476 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f16c8d-13a3-4625-9a90-c4744a5014cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:23:31.006253 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f16c8d-13a3-4625-9a90-c4744a5014cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:23:31.007824 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f16c8d-13a3-4625-9a90-c4744a5014cd None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:23:31.025639 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-98f16c8d-13a3-4625-9a90-c4744a5014cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:23:31.025639 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:23:31.025639 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:23:31.025639 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:23:31.025639 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:23:31.025639 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:23:31.025639 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:23:31.025639 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:23:31.025639 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:23:31.025639 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:23:31.025639 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:23:31.025639 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:23:31.025639 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 31 12:23:31.025639 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 31 12:23:31.025639 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 31 12:23:31.025639 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 31 12:23:31.026934 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:23:31.026934 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:23:31.026934 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:23:31.026934 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:23:31.026934 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:23:31.026934 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:23:31.026934 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:23:31.026934 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 31 12:23:31.026934 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:23:31.026934 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:23:31.026934 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:23:31.026934 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:23:31.026934 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:23:31.026934 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:23:31.026934 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:23:31.026934 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:23:31.026934 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 31 12:23:31.028384 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 31 12:23:31.028384 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 31 12:23:31.028384 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:23:31.028384 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 487/960] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Oct 31 12:23:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:23:31.040376 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f55246-7704-4490-904a-30e4df140b65 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:23:31.145429 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f55246-7704-4490-904a-30e4df140b65 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:23:31.160738 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:23:31.183594 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf07bc3-fa5a-484f-b7c1-0b4d44a8853c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:23:31.184225 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf07bc3-fa5a-484f-b7c1-0b4d44a8853c None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:23:31.184478 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf07bc3-fa5a-484f-b7c1-0b4d44a8853c None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:23:31.184745 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf07bc3-fa5a-484f-b7c1-0b4d44a8853c None admin] PATH_INFO: `/v3/domains` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:23:31.185387 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf07bc3-fa5a-484f-b7c1-0b4d44a8853c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:23:31.185682 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf07bc3-fa5a-484f-b7c1-0b4d44a8853c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:23:31.187045 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf07bc3-fa5a-484f-b7c1-0b4d44a8853c None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:23:31.199786 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 474/961] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Oct 31 12:23:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:23:31.205781 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f16c8d-13a3-4625-9a90-c4744a5014cd None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:23:31.243032 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:23:31.311194 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f16c8d-13a3-4625-9a90-c4744a5014cd None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:23:31.345373 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1417b782-80b6-41d7-a99e-5ec24119b3b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:23:31.346011 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1417b782-80b6-41d7-a99e-5ec24119b3b3 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:23:31.346283 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1417b782-80b6-41d7-a99e-5ec24119b3b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:23:31.346548 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1417b782-80b6-41d7-a99e-5ec24119b3b3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:23:31.354982 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1417b782-80b6-41d7-a99e-5ec24119b3b3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:23:31.356480 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1417b782-80b6-41d7-a99e-5ec24119b3b3 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:23:31.369796 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-1417b782-80b6-41d7-a99e-5ec24119b3b3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 31 12:23:31.369796 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:23:31.369796 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:23:31.369796 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:23:31.369796 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:23:31.369796 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:23:31.369796 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:23:31.369796 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:23:31.369796 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:23:31.369796 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:23:31.369796 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:23:31.369796 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:23:31.369796 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 31 12:23:31.369796 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 31 12:23:31.369796 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 31 12:23:31.370937 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 31 12:23:31.370937 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:23:31.370937 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:23:31.370937 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:23:31.370937 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:23:31.370937 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:23:31.370937 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:23:31.370937 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:23:31.370937 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 31 12:23:31.370937 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:23:31.370937 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:23:31.370937 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:23:31.370937 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:23:31.370937 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:23:31.370937 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:23:31.370937 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:23:31.370937 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:23:31.371874 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 31 12:23:31.371874 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 31 12:23:31.371874 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 31 12:23:31.371874 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 31 12:23:31.371874 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 31 12:23:31.371874 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 31 12:23:31.371874 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 31 12:23:31.371874 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 31 12:23:31.371874 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 488/962] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Oct 31 12:23:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:23:31.377078 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf07bc3-fa5a-484f-b7c1-0b4d44a8853c None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:23:31.481965 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf07bc3-fa5a-484f-b7c1-0b4d44a8853c None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:23:31.518892 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53714d2-9cb8-4a50-9314-1c728601236f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:23:31.519095 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53714d2-9cb8-4a50-9314-1c728601236f None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:23:31.519244 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53714d2-9cb8-4a50-9314-1c728601236f None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:23:31.519513 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53714d2-9cb8-4a50-9314-1c728601236f None admin] PATH_INFO: `/v3/projects` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:23:31.520196 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53714d2-9cb8-4a50-9314-1c728601236f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:23:31.520495 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53714d2-9cb8-4a50-9314-1c728601236f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:23:31.521962 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53714d2-9cb8-4a50-9314-1c728601236f None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:23:31.534026 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 475/963] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Oct 31 12:23:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 31 12:23:31.541036 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1417b782-80b6-41d7-a99e-5ec24119b3b3 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:23:31.638688 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:23:31.646553 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1417b782-80b6-41d7-a99e-5ec24119b3b3 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:23:31.680996 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb966c8-eefc-488c-9ef3-64fd67f06d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:23:31.681603 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb966c8-eefc-488c-9ef3-64fd67f06d6e None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:23:31.681872 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb966c8-eefc-488c-9ef3-64fd67f06d6e None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:23:31.682136 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb966c8-eefc-488c-9ef3-64fd67f06d6e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:23:31.682845 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb966c8-eefc-488c-9ef3-64fd67f06d6e None admin] RBAC: Adding query filter params (role.id=176c07b4fe3b4537bdfeb4be21b379c5, scope.project.id=2c32444cef584bce966b38203ce128b2, user.id=9e7777495d7a406c8c50b1ad52e47263) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:23:31.683144 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb966c8-eefc-488c-9ef3-64fd67f06d6e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:23:31.684566 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb966c8-eefc-488c-9ef3-64fd67f06d6e None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:23:31.690212 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 489/964] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Oct 31 12:23:31 2022] GET /identity/v3/role_assignments?user.id=9e7777495d7a406c8c50b1ad52e47263&scope.project.id=2c32444cef584bce966b38203ce128b2&role.id=176c07b4fe3b4537bdfeb4be21b379c5 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:23:33.809721 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e256c1fc-a7ce-4d58-88e8-265c0a682118 None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:23:33.810255 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e256c1fc-a7ce-4d58-88e8-265c0a682118 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:23:33.810255 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e256c1fc-a7ce-4d58-88e8-265c0a682118 None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:23:33.811638 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 476/965] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:23:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:23:33.816825 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfaf163-3c32-4d9d-9eec-04e56a57af9a None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:23:33.817242 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfaf163-3c32-4d9d-9eec-04e56a57af9a None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:23:33.817360 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfaf163-3c32-4d9d-9eec-04e56a57af9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:23:33.836260 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-8cfaf163-3c32-4d9d-9eec-04e56a57af9a None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:23:33.837421 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-8cfaf163-3c32-4d9d-9eec-04e56a57af9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:23:33.901365 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 490/966] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:23:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 31 12:23:34.172146 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e256c1fc-a7ce-4d58-88e8-265c0a682118 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:23:34.203383 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:23:34.272218 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e256c1fc-a7ce-4d58-88e8-265c0a682118 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:23:34.305760 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b7ae69-0b4b-45b5-8f81-f42436e75859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:23:34.306365 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b7ae69-0b4b-45b5-8f81-f42436e75859 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:23:34.306638 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b7ae69-0b4b-45b5-8f81-f42436e75859 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:23:34.306902 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b7ae69-0b4b-45b5-8f81-f42436e75859 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:23:34.307493 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b7ae69-0b4b-45b5-8f81-f42436e75859 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:23:34.308824 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b7ae69-0b4b-45b5-8f81-f42436e75859 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:23:34.317172 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-b6b7ae69-0b4b-45b5-8f81-f42436e75859 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 31 12:23:34.317172 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:23:34.317172 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 31 12:23:34.317172 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 31 12:23:34.317172 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 31 12:23:34.317172 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 31 12:23:34.317172 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 31 12:23:34.317172 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 31 12:23:34.317172 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 31 12:23:34.317172 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:23:34.317172 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 31 12:23:34.317172 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:23:34.317172 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:23:34.317172 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:23:34.317172 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:23:34.317172 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:23:34.317172 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:23:34.318422 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:23:34.318422 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:23:34.318422 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:23:34.318422 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:23:34.318422 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:23:34.318422 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:23:34.318422 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 31 12:23:34.318422 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 31 12:23:34.318422 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 31 12:23:34.318422 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 31 12:23:34.318422 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:23:34.318422 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:23:34.318422 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:23:34.318422 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:23:34.318422 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:23:34.318422 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:23:34.319294 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:23:34.319294 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:23:34.319294 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:23:34.319294 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:23:34.319294 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:23:34.319294 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:23:34.319294 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:23:34.319294 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:23:34.319294 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:23:34.319294 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:23:34.319294 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 31 12:23:34.319294 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 31 12:23:34.319294 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 31 12:23:34.319294 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:23:34.319294 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 477/967] 10.222.0.15 () {56 vars in 1117 bytes} [Mon Oct 31 12:23:34 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:23:34.324191 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfaf163-3c32-4d9d-9eec-04e56a57af9a None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:23:34.409348 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:23:34.419317 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfaf163-3c32-4d9d-9eec-04e56a57af9a None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:23:34.451336 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1416f7be-c8b5-4ea6-a229-38393bf117c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:23:34.451868 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1416f7be-c8b5-4ea6-a229-38393bf117c6 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:23:34.452139 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1416f7be-c8b5-4ea6-a229-38393bf117c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:23:34.452404 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1416f7be-c8b5-4ea6-a229-38393bf117c6 None admin] PATH_INFO: `/v3/services` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:23:34.453012 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1416f7be-c8b5-4ea6-a229-38393bf117c6 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:23:34.453310 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1416f7be-c8b5-4ea6-a229-38393bf117c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:23:34.454639 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1416f7be-c8b5-4ea6-a229-38393bf117c6 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:23:34.459413 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 491/968] 10.222.0.15 () {56 vars in 1116 bytes} [Mon Oct 31 12:23:34 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 31 12:23:34.465003 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b7ae69-0b4b-45b5-8f81-f42436e75859 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:23:34.556990 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:23:34.562906 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b7ae69-0b4b-45b5-8f81-f42436e75859 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:23:34.594961 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaaf2b1-c759-4fc7-92b0-5887aaf6f82b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:23:34.595592 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaaf2b1-c759-4fc7-92b0-5887aaf6f82b None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:23:34.595868 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaaf2b1-c759-4fc7-92b0-5887aaf6f82b None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:23:34.596134 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaaf2b1-c759-4fc7-92b0-5887aaf6f82b None admin] PATH_INFO: `/v3/services` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:23:34.596744 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaaf2b1-c759-4fc7-92b0-5887aaf6f82b None admin] RBAC: Adding query filter params (type=placement) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:23:34.597045 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaaf2b1-c759-4fc7-92b0-5887aaf6f82b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:23:34.598375 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaaf2b1-c759-4fc7-92b0-5887aaf6f82b None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:23:34.603440 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 478/969] 10.222.0.15 () {56 vars in 1116 bytes} [Mon Oct 31 12:23:34 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:23:36.745913 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58cb678-0de0-4e2d-9858-62c86220c7fd None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:23:36.746498 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58cb678-0de0-4e2d-9858-62c86220c7fd None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:23:36.746498 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58cb678-0de0-4e2d-9858-62c86220c7fd None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:23:36.747818 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 492/970] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:23:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:23:36.752993 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461ad4dc-3d2f-4c5f-97d7-c4c888ae44ff None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:23:36.753409 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461ad4dc-3d2f-4c5f-97d7-c4c888ae44ff None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:23:36.753531 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461ad4dc-3d2f-4c5f-97d7-c4c888ae44ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:23:36.772412 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-461ad4dc-3d2f-4c5f-97d7-c4c888ae44ff None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:23:36.773574 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-461ad4dc-3d2f-4c5f-97d7-c4c888ae44ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:23:36.837535 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 479/971] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:23:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 31 12:23:37.115853 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58cb678-0de0-4e2d-9858-62c86220c7fd None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:23:37.214314 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58cb678-0de0-4e2d-9858-62c86220c7fd None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:23:37.246476 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dbd27a-0025-4c59-89bf-2cab849f989b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:23:37.247065 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dbd27a-0025-4c59-89bf-2cab849f989b None admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:23:37.247366 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dbd27a-0025-4c59-89bf-2cab849f989b None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:23:37.247635 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dbd27a-0025-4c59-89bf-2cab849f989b None admin] PATH_INFO: `/v3/services` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:23:37.248271 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dbd27a-0025-4c59-89bf-2cab849f989b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:23:37.249607 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dbd27a-0025-4c59-89bf-2cab849f989b None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:23:37.264199 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 493/972] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Oct 31 12:23:37 2022] POST /identity/v3/services => generated 246 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:23:39.424945 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44ff7e5-1be4-422b-ac74-3059687d35c5 None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:23:39.425466 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44ff7e5-1be4-422b-ac74-3059687d35c5 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:23:39.425466 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44ff7e5-1be4-422b-ac74-3059687d35c5 None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:23:39.426731 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 480/973] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:23:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:23:39.432633 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f17dc1d-033d-4fcf-a27c-5728bce6018f None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:23:39.433054 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f17dc1d-033d-4fcf-a27c-5728bce6018f None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:23:39.433177 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f17dc1d-033d-4fcf-a27c-5728bce6018f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:23:39.451410 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-9f17dc1d-033d-4fcf-a27c-5728bce6018f None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:23:39.452578 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-9f17dc1d-033d-4fcf-a27c-5728bce6018f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:23:39.461579 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:23:39.524730 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 494/974] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:23:39 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 31 12:23:39.799679 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44ff7e5-1be4-422b-ac74-3059687d35c5 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:23:39.887095 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:23:39.898448 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44ff7e5-1be4-422b-ac74-3059687d35c5 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:23:39.930700 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945f81c9-772c-4f9a-bc10-6104bf41b312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:23:39.931337 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945f81c9-772c-4f9a-bc10-6104bf41b312 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:23:39.931612 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945f81c9-772c-4f9a-bc10-6104bf41b312 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:23:39.931878 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945f81c9-772c-4f9a-bc10-6104bf41b312 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:23:39.932440 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945f81c9-772c-4f9a-bc10-6104bf41b312 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:23:39.933776 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945f81c9-772c-4f9a-bc10-6104bf41b312 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:23:39.942266 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-945f81c9-772c-4f9a-bc10-6104bf41b312 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 31 12:23:39.942266 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:23:39.942266 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 31 12:23:39.942266 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 31 12:23:39.942266 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 31 12:23:39.942266 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 31 12:23:39.942266 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 31 12:23:39.942266 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 31 12:23:39.942266 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 31 12:23:39.942266 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:23:39.942266 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 31 12:23:39.942266 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:23:39.942266 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:23:39.942266 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:23:39.942266 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:23:39.942266 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:23:39.942266 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:23:39.944609 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:23:39.944609 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:23:39.944609 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:23:39.944609 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:23:39.944609 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:23:39.944609 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:23:39.944609 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 31 12:23:39.944609 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 31 12:23:39.944609 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 31 12:23:39.944609 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 31 12:23:39.944609 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:23:39.944609 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:23:39.944609 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:23:39.944609 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:23:39.944609 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:23:39.944609 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:23:39.945730 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:23:39.945730 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:23:39.945730 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:23:39.945730 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:23:39.945730 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:23:39.945730 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:23:39.945730 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:23:39.945730 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:23:39.945730 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:23:39.945730 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:23:39.945730 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 31 12:23:39.945730 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 31 12:23:39.945730 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 31 12:23:39.945730 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:23:39.945730 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 481/975] 10.222.0.15 () {56 vars in 1117 bytes} [Mon Oct 31 12:23:39 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:23:39.949931 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f17dc1d-033d-4fcf-a27c-5728bce6018f None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:23:40.047785 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f17dc1d-033d-4fcf-a27c-5728bce6018f None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:23:40.081233 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d42657a-c0b4-4397-8c49-4ea408edf96b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:23:40.081778 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d42657a-c0b4-4397-8c49-4ea408edf96b None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:23:40.082058 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d42657a-c0b4-4397-8c49-4ea408edf96b None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:23:40.082331 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d42657a-c0b4-4397-8c49-4ea408edf96b None admin] PATH_INFO: `/v3/services` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:23:40.082946 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d42657a-c0b4-4397-8c49-4ea408edf96b None admin] RBAC: Adding query filter params (name=placement) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:23:40.083361 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d42657a-c0b4-4397-8c49-4ea408edf96b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:23:40.084713 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d42657a-c0b4-4397-8c49-4ea408edf96b None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:23:40.089663 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 495/976] 10.222.0.15 () {56 vars in 1116 bytes} [Mon Oct 31 12:23:39 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:23:40.095604 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945f81c9-772c-4f9a-bc10-6104bf41b312 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:23:40.191466 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945f81c9-772c-4f9a-bc10-6104bf41b312 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:23:40.224566 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8f0244-af2c-4b96-b698-2fa6d9a77894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:23:40.225145 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8f0244-af2c-4b96-b698-2fa6d9a77894 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:23:40.225416 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8f0244-af2c-4b96-b698-2fa6d9a77894 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:23:40.225677 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8f0244-af2c-4b96-b698-2fa6d9a77894 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:23:40.226354 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8f0244-af2c-4b96-b698-2fa6d9a77894 None admin] RBAC: Adding query filter params (interface=public, service_id=5f9ff462078a46348b5e1bd547589519, region_id=RegionOne) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:23:40.226654 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8f0244-af2c-4b96-b698-2fa6d9a77894 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:23:40.228020 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8f0244-af2c-4b96-b698-2fa6d9a77894 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:23:40.232809 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 482/977] 10.222.0.15 () {56 vars in 1251 bytes} [Mon Oct 31 12:23:40 2022] GET /identity/v3/endpoints?service_id=5f9ff462078a46348b5e1bd547589519&interface=public®ion_id=RegionOne => generated 194 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:23:40.238386 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d42657a-c0b4-4397-8c49-4ea408edf96b None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:23:40.242212 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:23:40.335977 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d42657a-c0b4-4397-8c49-4ea408edf96b None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:23:40.370605 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a052f98-e80e-419c-a310-b0e2c23096b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:23:40.371293 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a052f98-e80e-419c-a310-b0e2c23096b8 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:23:40.371578 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a052f98-e80e-419c-a310-b0e2c23096b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:23:40.371847 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a052f98-e80e-419c-a310-b0e2c23096b8 None admin] PATH_INFO: `/v3/services` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:23:40.372439 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a052f98-e80e-419c-a310-b0e2c23096b8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:23:40.373778 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a052f98-e80e-419c-a310-b0e2c23096b8 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:23:40.380605 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 496/978] 10.222.0.15 () {56 vars in 1087 bytes} [Mon Oct 31 12:23:40 2022] GET /identity/v3/services => generated 2191 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:23:42.567673 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff82f8a4-55df-446f-a0c3-0665158c3d61 None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:23:42.568216 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff82f8a4-55df-446f-a0c3-0665158c3d61 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:23:42.568216 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff82f8a4-55df-446f-a0c3-0665158c3d61 None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:23:42.569556 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 483/979] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:23:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:23:42.574667 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822c3f8f-5446-4c7e-971f-892aa9d06dd7 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:23:42.575052 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822c3f8f-5446-4c7e-971f-892aa9d06dd7 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:23:42.575251 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822c3f8f-5446-4c7e-971f-892aa9d06dd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:23:42.592769 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-822c3f8f-5446-4c7e-971f-892aa9d06dd7 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:23:42.593930 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-822c3f8f-5446-4c7e-971f-892aa9d06dd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:23:42.653705 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:23:42.659282 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 497/980] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:23:42 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 31 12:23:42.936643 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff82f8a4-55df-446f-a0c3-0665158c3d61 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:23:42.955827 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:23:43.037198 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff82f8a4-55df-446f-a0c3-0665158c3d61 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:23:43.072181 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72e4fcc-5e63-4e70-979e-540ea4c99236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:23:43.072796 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72e4fcc-5e63-4e70-979e-540ea4c99236 None admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:23:43.073071 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72e4fcc-5e63-4e70-979e-540ea4c99236 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:23:43.073339 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72e4fcc-5e63-4e70-979e-540ea4c99236 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:23:43.073942 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72e4fcc-5e63-4e70-979e-540ea4c99236 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:23:43.075323 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72e4fcc-5e63-4e70-979e-540ea4c99236 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:23:43.084269 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application [None req-a72e4fcc-5e63-4e70-979e-540ea4c99236 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 31 12:23:43.084269 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:23:43.084269 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 31 12:23:43.084269 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 31 12:23:43.084269 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 31 12:23:43.084269 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 31 12:23:43.084269 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 31 12:23:43.084269 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 31 12:23:43.084269 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 31 12:23:43.084269 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:23:43.084269 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 31 12:23:43.084269 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:23:43.084269 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 31 12:23:43.084269 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 31 12:23:43.084269 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 31 12:23:43.084269 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 31 12:23:43.084269 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 31 12:23:43.085484 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 31 12:23:43.085484 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 31 12:23:43.085484 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 31 12:23:43.085484 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 31 12:23:43.085484 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 31 12:23:43.085484 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 31 12:23:43.085484 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 31 12:23:43.085484 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 31 12:23:43.085484 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 31 12:23:43.085484 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 31 12:23:43.085484 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 31 12:23:43.085484 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 31 12:23:43.085484 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 31 12:23:43.085484 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 31 12:23:43.085484 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 31 12:23:43.085484 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.get_or_create( Oct 31 12:23:43.086524 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 31 12:23:43.086524 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application with Lock( Oct 31 12:23:43.086524 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 31 12:23:43.086524 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self._enter() Oct 31 12:23:43.086524 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 31 12:23:43.086524 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 31 12:23:43.086524 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 31 12:23:43.086524 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application return self.creator() Oct 31 12:23:43.086524 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 31 12:23:43.086524 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application created_value = creator( Oct 31 12:23:43.086524 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 31 12:23:43.086524 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 31 12:23:43.086524 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 31 12:23:43.086524 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: ERROR keystone.server.flask.application Oct 31 12:23:43.086524 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 484/981] 10.222.0.15 () {56 vars in 1117 bytes} [Mon Oct 31 12:23:42 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 31 12:23:43.091350 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822c3f8f-5446-4c7e-971f-892aa9d06dd7 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:23:43.187311 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822c3f8f-5446-4c7e-971f-892aa9d06dd7 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:23:43.219424 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a17b7a-2aec-4a36-a321-f80fb6441d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:23:43.220012 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a17b7a-2aec-4a36-a321-f80fb6441d45 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:23:43.220284 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a17b7a-2aec-4a36-a321-f80fb6441d45 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:23:43.220548 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a17b7a-2aec-4a36-a321-f80fb6441d45 None admin] PATH_INFO: `/v3/services` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:23:43.221158 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a17b7a-2aec-4a36-a321-f80fb6441d45 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:23:43.221454 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a17b7a-2aec-4a36-a321-f80fb6441d45 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:23:43.222780 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a17b7a-2aec-4a36-a321-f80fb6441d45 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:23:43.228384 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 498/982] 10.222.0.15 () {56 vars in 1116 bytes} [Mon Oct 31 12:23:43 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:23:43.234064 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72e4fcc-5e63-4e70-979e-540ea4c99236 None admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:23:43.313986 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:23:43.332195 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72e4fcc-5e63-4e70-979e-540ea4c99236 None admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:23:43.364908 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c2eb7c-6846-488d-bf3a-97e50a9bb544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:23:43.365489 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c2eb7c-6846-488d-bf3a-97e50a9bb544 None admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:23:43.365763 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c2eb7c-6846-488d-bf3a-97e50a9bb544 None admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:23:43.366029 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c2eb7c-6846-488d-bf3a-97e50a9bb544 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:23:43.366682 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c2eb7c-6846-488d-bf3a-97e50a9bb544 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:23:43.368045 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c2eb7c-6846-488d-bf3a-97e50a9bb544 None admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:23:43.386944 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 485/983] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:23:43 2022] POST /identity/v3/endpoints => generated 331 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:23:53.225460 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=69 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:23:53.382868 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:24:47.042302 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265e5cef-9b54-4ff3-91e1-772ffd13e4e0 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:24:47.042729 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265e5cef-9b54-4ff3-91e1-772ffd13e4e0 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:24:47.042831 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265e5cef-9b54-4ff3-91e1-772ffd13e4e0 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:24:47.049430 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134529db-952e-47b1-9915-9c10576b9cfe None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:24:47.049836 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134529db-952e-47b1-9915-9c10576b9cfe None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:24:47.050349 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134529db-952e-47b1-9915-9c10576b9cfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:24:47.054973 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 499/984] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:24:47 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:24:47.180806 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-134529db-952e-47b1-9915-9c10576b9cfe None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:24:47.182263 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-134529db-952e-47b1-9915-9c10576b9cfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:24:47.318722 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 486/985] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:24:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:24:47.621780 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a8a58c-e1d4-4dc4-9e21-d4cfec00c9c8 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:24:47.622414 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a8a58c-e1d4-4dc4-9e21-d4cfec00c9c8 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:24:47.622880 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a8a58c-e1d4-4dc4-9e21-d4cfec00c9c8 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:24:47.624734 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 500/986] 10.222.0.15 () {54 vars in 965 bytes} [Mon Oct 31 12:24:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:24:47.631312 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8152530c-a07f-433b-8609-6ccce6e728af None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:24:47.631857 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8152530c-a07f-433b-8609-6ccce6e728af None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:24:47.632333 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8152530c-a07f-433b-8609-6ccce6e728af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:24:47.712412 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-8152530c-a07f-433b-8609-6ccce6e728af None None] MFA Rules not processed for user `25e9c7d1e4714bf18a1ced84855423d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:24:47.713962 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-8152530c-a07f-433b-8609-6ccce6e728af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:24:47.736268 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:24:47.795114 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 487/987] 10.222.0.15 () {58 vars in 1062 bytes} [Mon Oct 31 12:24:47 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:24:47.805291 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a8a58c-e1d4-4dc4-9e21-d4cfec00c9c8 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:24:47.807221 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-97a8a58c-e1d4-4dc4-9e21-d4cfec00c9c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:24:47.936857 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a8a58c-e1d4-4dc4-9e21-d4cfec00c9c8 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:24:48.031490 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b195015-dff9-41c1-9940-ba5816569e58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:24:48.032110 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b195015-dff9-41c1-9940-ba5816569e58 service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:24:48.032387 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b195015-dff9-41c1-9940-ba5816569e58 service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:24:48.032654 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b195015-dff9-41c1-9940-ba5816569e58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:24:48.034933 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-8b195015-dff9-41c1-9940-ba5816569e58 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:24:48.092982 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:24:48.162102 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b195015-dff9-41c1-9940-ba5816569e58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:24:48.163584 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b195015-dff9-41c1-9940-ba5816569e58 service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:24:48.187532 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 501/988] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:24:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:24:50.760434 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100c493d-0937-466f-a856-e3dd06116be7 None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:24:50.760955 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100c493d-0937-466f-a856-e3dd06116be7 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:24:50.760955 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100c493d-0937-466f-a856-e3dd06116be7 None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:24:50.762267 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 488/989] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:24:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:24:50.769149 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191cfcc4-9809-4a4e-9a7c-c4afda642f52 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:24:50.769677 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191cfcc4-9809-4a4e-9a7c-c4afda642f52 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:24:50.769933 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191cfcc4-9809-4a4e-9a7c-c4afda642f52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:24:50.853301 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-191cfcc4-9809-4a4e-9a7c-c4afda642f52 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:24:50.854505 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-191cfcc4-9809-4a4e-9a7c-c4afda642f52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:24:50.951677 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 502/990] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:24:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:24:51.238463 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100c493d-0937-466f-a856-e3dd06116be7 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:24:51.239999 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-100c493d-0937-466f-a856-e3dd06116be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:24:51.345077 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100c493d-0937-466f-a856-e3dd06116be7 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:24:51.415748 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:24:51.418007 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41fc1a5-16f3-476a-81e4-bbb87c7c5292 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:24:51.418646 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41fc1a5-16f3-476a-81e4-bbb87c7c5292 service neutron] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:24:51.419106 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41fc1a5-16f3-476a-81e4-bbb87c7c5292 service neutron] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:24:51.419574 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41fc1a5-16f3-476a-81e4-bbb87c7c5292 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:24:51.421369 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-e41fc1a5-16f3-476a-81e4-bbb87c7c5292 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:24:51.526811 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41fc1a5-16f3-476a-81e4-bbb87c7c5292 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:24:51.528251 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41fc1a5-16f3-476a-81e4-bbb87c7c5292 service neutron] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:24:51.546249 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 489/991] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:24:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:24:53.996294 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7982a23-8aeb-42ae-abef-8caff9796e70 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:24:53.996853 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7982a23-8aeb-42ae-abef-8caff9796e70 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:24:53.996853 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7982a23-8aeb-42ae-abef-8caff9796e70 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:24:53.998316 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 503/992] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:24:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:24:54.004144 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f69603b-fde2-4a3b-9b37-a46187881677 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:24:54.004541 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f69603b-fde2-4a3b-9b37-a46187881677 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:24:54.004687 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f69603b-fde2-4a3b-9b37-a46187881677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:24:54.075297 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-2f69603b-fde2-4a3b-9b37-a46187881677 None None] MFA Rules not processed for user `d5915aed320e40a1ba7723089db18636`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:24:54.076506 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-2f69603b-fde2-4a3b-9b37-a46187881677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:24:54.128393 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:24:54.172306 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 490/993] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:24:54 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:24:54.462121 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7982a23-8aeb-42ae-abef-8caff9796e70 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:24:54.463659 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-f7982a23-8aeb-42ae-abef-8caff9796e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:24:54.473809 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:24:54.564394 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7982a23-8aeb-42ae-abef-8caff9796e70 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:24:54.636731 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77f0ef5-c18d-43c6-87c7-d061a288eee1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:24:54.637327 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77f0ef5-c18d-43c6-87c7-d061a288eee1 service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:24:54.637598 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77f0ef5-c18d-43c6-87c7-d061a288eee1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:24:54.637860 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77f0ef5-c18d-43c6-87c7-d061a288eee1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:24:54.639443 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-e77f0ef5-c18d-43c6-87c7-d061a288eee1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:24:54.702082 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:24:54.747554 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77f0ef5-c18d-43c6-87c7-d061a288eee1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:24:54.748904 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77f0ef5-c18d-43c6-87c7-d061a288eee1 service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:24:54.766538 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 504/994] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:24:54 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 31 12:24:57.439746 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2137a8ba-114e-4cf4-a1de-cc30444bf831 None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:24:57.440342 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2137a8ba-114e-4cf4-a1de-cc30444bf831 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:24:57.440342 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2137a8ba-114e-4cf4-a1de-cc30444bf831 None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:24:57.441787 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 491/995] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:24:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:24:57.447151 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f236a6df-b016-4a6c-ad92-f013f6b2166b None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:24:57.447559 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f236a6df-b016-4a6c-ad92-f013f6b2166b None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:24:57.447723 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f236a6df-b016-4a6c-ad92-f013f6b2166b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:24:57.512405 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-f236a6df-b016-4a6c-ad92-f013f6b2166b None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:24:57.513584 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-f236a6df-b016-4a6c-ad92-f013f6b2166b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:24:57.607355 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 505/996] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:24:57 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:24:57.901762 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2137a8ba-114e-4cf4-a1de-cc30444bf831 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:24:57.903349 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-2137a8ba-114e-4cf4-a1de-cc30444bf831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:24:58.005264 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2137a8ba-114e-4cf4-a1de-cc30444bf831 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:24:58.077990 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7c7b49-03fa-4c25-aa40-3bdfbdc87351 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:24:58.078597 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7c7b49-03fa-4c25-aa40-3bdfbdc87351 service neutron] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:24:58.078868 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7c7b49-03fa-4c25-aa40-3bdfbdc87351 service neutron] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:24:58.079133 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7c7b49-03fa-4c25-aa40-3bdfbdc87351 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:24:58.081260 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-ca7c7b49-03fa-4c25-aa40-3bdfbdc87351 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:24:58.088024 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:24:58.189043 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7c7b49-03fa-4c25-aa40-3bdfbdc87351 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:24:58.190398 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7c7b49-03fa-4c25-aa40-3bdfbdc87351 service neutron] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:24:58.208482 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 492/997] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:24:57 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:25:01.003976 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1663e123-5c50-4f31-902a-9f4ec3fdaead None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:25:01.004542 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1663e123-5c50-4f31-902a-9f4ec3fdaead None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:25:01.004542 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1663e123-5c50-4f31-902a-9f4ec3fdaead None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:25:01.005846 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 506/998] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:25:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:25:01.011048 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21caf72-f299-4772-83c1-57a44db28319 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:25:01.011483 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21caf72-f299-4772-83c1-57a44db28319 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:25:01.011630 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21caf72-f299-4772-83c1-57a44db28319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:25:01.089084 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-a21caf72-f299-4772-83c1-57a44db28319 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:25:01.090328 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-a21caf72-f299-4772-83c1-57a44db28319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:25:01.154701 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:25:01.198056 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 493/999] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:25:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:25:01.493516 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1663e123-5c50-4f31-902a-9f4ec3fdaead None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:25:01.495036 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-1663e123-5c50-4f31-902a-9f4ec3fdaead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:25:01.515301 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:25:01.599357 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1663e123-5c50-4f31-902a-9f4ec3fdaead None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:25:01.673976 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d662f8c5-f2b9-444d-a762-d801d972cdac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:25:01.675409 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d662f8c5-f2b9-444d-a762-d801d972cdac service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:25:01.675707 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d662f8c5-f2b9-444d-a762-d801d972cdac service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:25:01.675977 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d662f8c5-f2b9-444d-a762-d801d972cdac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:25:01.677560 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-d662f8c5-f2b9-444d-a762-d801d972cdac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:25:01.749131 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:25:01.787305 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d662f8c5-f2b9-444d-a762-d801d972cdac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:25:01.788750 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d662f8c5-f2b9-444d-a762-d801d972cdac service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:25:01.807272 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 507/1000] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:25:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:25:04.004705 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fd42a3-c106-49f6-a372-462861494260 None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:25:04.006229 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fd42a3-c106-49f6-a372-462861494260 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:25:04.006229 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fd42a3-c106-49f6-a372-462861494260 None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:25:04.007028 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 494/1001] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:25:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:25:04.012930 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c023401-0132-4522-8bb4-f69b0d92c71c None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:25:04.013422 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c023401-0132-4522-8bb4-f69b0d92c71c None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:25:04.013515 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c023401-0132-4522-8bb4-f69b0d92c71c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:25:04.094867 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-5c023401-0132-4522-8bb4-f69b0d92c71c None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:25:04.096559 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-5c023401-0132-4522-8bb4-f69b0d92c71c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:25:04.215350 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 508/1002] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:25:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:25:04.501294 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fd42a3-c106-49f6-a372-462861494260 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:25:04.504307 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-19fd42a3-c106-49f6-a372-462861494260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:25:04.608183 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fd42a3-c106-49f6-a372-462861494260 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:25:04.682795 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77e52e1-db17-4e04-9b7f-82cc1e3e88a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:25:04.683429 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77e52e1-db17-4e04-9b7f-82cc1e3e88a6 service neutron] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:25:04.683709 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77e52e1-db17-4e04-9b7f-82cc1e3e88a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:25:04.683984 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77e52e1-db17-4e04-9b7f-82cc1e3e88a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:25:04.685546 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-f77e52e1-db17-4e04-9b7f-82cc1e3e88a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:25:04.703261 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:25:04.795273 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77e52e1-db17-4e04-9b7f-82cc1e3e88a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:25:04.796705 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77e52e1-db17-4e04-9b7f-82cc1e3e88a6 service neutron] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:25:04.815994 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 495/1003] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:25:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:25:07.026721 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a0455f-d554-4ed8-8781-1157bab6980a None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:25:07.027202 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a0455f-d554-4ed8-8781-1157bab6980a None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:25:07.027291 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a0455f-d554-4ed8-8781-1157bab6980a None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:25:07.028644 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 509/1004] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:25:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:25:07.034332 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214b80d9-ac89-4f00-ad33-86ef848dd68b None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:25:07.034970 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214b80d9-ac89-4f00-ad33-86ef848dd68b None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:25:07.034970 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214b80d9-ac89-4f00-ad33-86ef848dd68b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:25:07.109378 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-214b80d9-ac89-4f00-ad33-86ef848dd68b None None] MFA Rules not processed for user `d5915aed320e40a1ba7723089db18636`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:25:07.110616 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-214b80d9-ac89-4f00-ad33-86ef848dd68b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:25:07.179536 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:25:07.213574 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 496/1005] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:25:07 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:25:07.503748 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a0455f-d554-4ed8-8781-1157bab6980a None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:25:07.505312 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-93a0455f-d554-4ed8-8781-1157bab6980a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:25:07.538620 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:25:07.616553 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a0455f-d554-4ed8-8781-1157bab6980a None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:25:07.692322 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c71f90-438e-46b1-a32a-6c28cee3fbf6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:25:07.692848 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c71f90-438e-46b1-a32a-6c28cee3fbf6 service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:25:07.693125 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c71f90-438e-46b1-a32a-6c28cee3fbf6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:25:07.693396 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c71f90-438e-46b1-a32a-6c28cee3fbf6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:25:07.694966 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-61c71f90-438e-46b1-a32a-6c28cee3fbf6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:25:07.774662 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:25:07.805486 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c71f90-438e-46b1-a32a-6c28cee3fbf6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:25:07.806906 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c71f90-438e-46b1-a32a-6c28cee3fbf6 service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:25:07.825427 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 510/1006] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:25:07 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:25:10.221711 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c22f586-2c61-4da8-b9c8-09d69846b29a None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:25:10.222258 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c22f586-2c61-4da8-b9c8-09d69846b29a None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:25:10.222258 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c22f586-2c61-4da8-b9c8-09d69846b29a None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:25:10.223665 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 497/1007] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:25:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:25:10.228952 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dba10d-d743-4ce1-8ba9-1b4fb04ca604 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:25:10.229388 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dba10d-d743-4ce1-8ba9-1b4fb04ca604 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:25:10.229518 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dba10d-d743-4ce1-8ba9-1b4fb04ca604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:25:10.294895 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-f1dba10d-d743-4ce1-8ba9-1b4fb04ca604 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:25:10.296030 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-f1dba10d-d743-4ce1-8ba9-1b4fb04ca604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:25:10.391796 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 511/1008] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:25:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:25:10.673763 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c22f586-2c61-4da8-b9c8-09d69846b29a None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:25:10.675804 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-0c22f586-2c61-4da8-b9c8-09d69846b29a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:25:10.777066 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c22f586-2c61-4da8-b9c8-09d69846b29a None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:25:10.850290 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766f559d-8bb1-4ab5-91e8-b9cc5eda3b05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:25:10.850891 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766f559d-8bb1-4ab5-91e8-b9cc5eda3b05 service neutron] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:25:10.851191 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766f559d-8bb1-4ab5-91e8-b9cc5eda3b05 service neutron] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:25:10.851460 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766f559d-8bb1-4ab5-91e8-b9cc5eda3b05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:25:10.853017 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-766f559d-8bb1-4ab5-91e8-b9cc5eda3b05 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:25:10.880218 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:25:10.960402 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766f559d-8bb1-4ab5-91e8-b9cc5eda3b05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:25:10.961747 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766f559d-8bb1-4ab5-91e8-b9cc5eda3b05 service neutron] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:25:10.980283 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 498/1009] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:25:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:25:13.168541 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c0efce-6ad4-44f5-8ab7-e83d46ee7592 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:25:13.169126 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c0efce-6ad4-44f5-8ab7-e83d46ee7592 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:25:13.169126 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c0efce-6ad4-44f5-8ab7-e83d46ee7592 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:25:13.170409 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 512/1010] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:25:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:25:13.176292 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa1512c-3d1e-4717-b22c-b8d610ff6a6f None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:25:13.176711 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa1512c-3d1e-4717-b22c-b8d610ff6a6f None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:25:13.176973 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa1512c-3d1e-4717-b22c-b8d610ff6a6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:25:13.242589 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-9fa1512c-3d1e-4717-b22c-b8d610ff6a6f None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:25:13.243814 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-9fa1512c-3d1e-4717-b22c-b8d610ff6a6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:25:13.316898 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:25:13.339498 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 499/1011] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:25:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:25:13.625098 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c0efce-6ad4-44f5-8ab7-e83d46ee7592 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:25:13.626607 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-16c0efce-6ad4-44f5-8ab7-e83d46ee7592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:25:13.666778 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:25:13.730227 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c0efce-6ad4-44f5-8ab7-e83d46ee7592 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:25:13.801335 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f755b49-78b3-4a70-a75b-1d7053efe4cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:25:13.801931 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f755b49-78b3-4a70-a75b-1d7053efe4cf service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:25:13.802206 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f755b49-78b3-4a70-a75b-1d7053efe4cf service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:25:13.802471 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f755b49-78b3-4a70-a75b-1d7053efe4cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:25:13.804066 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-0f755b49-78b3-4a70-a75b-1d7053efe4cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:25:13.885957 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:25:13.914442 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f755b49-78b3-4a70-a75b-1d7053efe4cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:25:13.915918 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f755b49-78b3-4a70-a75b-1d7053efe4cf service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:25:13.934259 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 513/1012] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:25:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:25:16.392969 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50d15f2-5076-4691-8ae7-23d75d3e1c96 None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:25:16.393514 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50d15f2-5076-4691-8ae7-23d75d3e1c96 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:25:16.393835 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50d15f2-5076-4691-8ae7-23d75d3e1c96 None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:25:16.395279 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 500/1013] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:25:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:25:16.400379 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5fa331-228c-459f-ad9f-a331df96b0de None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:25:16.400779 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5fa331-228c-459f-ad9f-a331df96b0de None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:25:16.400923 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5fa331-228c-459f-ad9f-a331df96b0de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:25:16.465007 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-8b5fa331-228c-459f-ad9f-a331df96b0de None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:25:16.466205 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-8b5fa331-228c-459f-ad9f-a331df96b0de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:25:16.561983 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 514/1014] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:25:16 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:25:16.848571 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50d15f2-5076-4691-8ae7-23d75d3e1c96 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:25:16.850129 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-f50d15f2-5076-4691-8ae7-23d75d3e1c96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:25:16.951513 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50d15f2-5076-4691-8ae7-23d75d3e1c96 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:25:17.027312 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ddbf2d-6b46-4a85-83d3-428722463a14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:25:17.027915 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ddbf2d-6b46-4a85-83d3-428722463a14 service neutron] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:25:17.028191 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ddbf2d-6b46-4a85-83d3-428722463a14 service neutron] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:25:17.028457 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ddbf2d-6b46-4a85-83d3-428722463a14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:25:17.030024 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-a7ddbf2d-6b46-4a85-83d3-428722463a14 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:25:17.066477 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:25:17.138703 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ddbf2d-6b46-4a85-83d3-428722463a14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:25:17.140086 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ddbf2d-6b46-4a85-83d3-428722463a14 service neutron] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:25:17.158491 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 501/1015] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:25:16 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:25:21.182042 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748bd041-ef61-437b-bab4-62996ad84ab2 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:25:21.182565 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748bd041-ef61-437b-bab4-62996ad84ab2 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:25:21.182565 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748bd041-ef61-437b-bab4-62996ad84ab2 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:25:21.183963 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 515/1016] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:25:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:25:21.190068 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1029b1-9c54-4ced-bedb-379b5be00dee None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:25:21.190488 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1029b1-9c54-4ced-bedb-379b5be00dee None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:25:21.190596 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1029b1-9c54-4ced-bedb-379b5be00dee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:25:21.256401 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-ac1029b1-9c54-4ced-bedb-379b5be00dee None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:25:21.257491 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-ac1029b1-9c54-4ced-bedb-379b5be00dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:25:21.347335 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:25:21.359459 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 502/1017] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:25:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:25:21.644703 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748bd041-ef61-437b-bab4-62996ad84ab2 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:25:21.646220 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-748bd041-ef61-437b-bab4-62996ad84ab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:25:21.694625 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:25:21.753709 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748bd041-ef61-437b-bab4-62996ad84ab2 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:25:21.827664 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efcddb3-e3fa-4379-977d-1d1225dc0351 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:25:21.828272 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efcddb3-e3fa-4379-977d-1d1225dc0351 service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:25:21.828549 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efcddb3-e3fa-4379-977d-1d1225dc0351 service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:25:21.828817 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efcddb3-e3fa-4379-977d-1d1225dc0351 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:25:21.831353 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-1efcddb3-e3fa-4379-977d-1d1225dc0351 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:25:21.924636 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:25:21.941415 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efcddb3-e3fa-4379-977d-1d1225dc0351 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:25:21.942831 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efcddb3-e3fa-4379-977d-1d1225dc0351 service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:25:21.961972 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 516/1018] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:25:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:25:24.770408 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df059a08-15b0-4241-b341-42b5b9175264 None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:25:24.770917 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df059a08-15b0-4241-b341-42b5b9175264 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:25:24.770917 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df059a08-15b0-4241-b341-42b5b9175264 None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:25:24.772221 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 503/1019] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:25:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:25:24.777320 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80c2d0f-4ee0-4672-9f95-72e9e3402f43 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:25:24.777750 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80c2d0f-4ee0-4672-9f95-72e9e3402f43 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:25:24.777841 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80c2d0f-4ee0-4672-9f95-72e9e3402f43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:25:24.843507 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-a80c2d0f-4ee0-4672-9f95-72e9e3402f43 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:25:24.844690 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-a80c2d0f-4ee0-4672-9f95-72e9e3402f43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:25:24.942249 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 517/1020] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:25:24 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:25:25.245077 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17217303-8d52-4a0e-89ae-429df6f9a978 None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:25:25.245359 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17217303-8d52-4a0e-89ae-429df6f9a978 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:25:25.245632 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17217303-8d52-4a0e-89ae-429df6f9a978 None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:25:25.247003 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 504/1021] 10.222.0.15 () {54 vars in 965 bytes} [Mon Oct 31 12:25:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:25:25.253499 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdad61f-20ef-4392-9c31-063da3abcfd7 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:25:25.253863 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdad61f-20ef-4392-9c31-063da3abcfd7 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:25:25.254144 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdad61f-20ef-4392-9c31-063da3abcfd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:25:25.326921 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:25:25.337219 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-2cdad61f-20ef-4392-9c31-063da3abcfd7 None None] MFA Rules not processed for user `25e9c7d1e4714bf18a1ced84855423d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:25:25.338433 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-2cdad61f-20ef-4392-9c31-063da3abcfd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:25:25.415851 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 518/1022] 10.222.0.15 () {58 vars in 1062 bytes} [Mon Oct 31 12:25:25 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 31 12:25:25.425761 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17217303-8d52-4a0e-89ae-429df6f9a978 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:25:25.427381 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-17217303-8d52-4a0e-89ae-429df6f9a978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:25:25.536851 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17217303-8d52-4a0e-89ae-429df6f9a978 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:25:25.610393 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7968cbe5-83db-4514-9365-6a07a42e08b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:25:25.611551 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7968cbe5-83db-4514-9365-6a07a42e08b4 service neutron] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:25:25.611828 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7968cbe5-83db-4514-9365-6a07a42e08b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:25:25.612096 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7968cbe5-83db-4514-9365-6a07a42e08b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:25:25.613662 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-7968cbe5-83db-4514-9365-6a07a42e08b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:25:25.670302 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:25:25.758722 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7968cbe5-83db-4514-9365-6a07a42e08b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:25:25.761603 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7968cbe5-83db-4514-9365-6a07a42e08b4 service neutron] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:25:25.779874 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 505/1023] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:25:25 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:25:30.674973 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d469603c-9ced-4f14-aa16-cdd2a7e764d7 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:25:30.675507 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d469603c-9ced-4f14-aa16-cdd2a7e764d7 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:25:30.675600 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d469603c-9ced-4f14-aa16-cdd2a7e764d7 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:25:30.676875 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 519/1024] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:25:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:25:30.682181 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c8900b-bb51-4ed4-b767-42dc8fc04f5b None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:25:30.682686 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c8900b-bb51-4ed4-b767-42dc8fc04f5b None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:25:30.682861 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c8900b-bb51-4ed4-b767-42dc8fc04f5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:25:30.753507 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-48c8900b-bb51-4ed4-b767-42dc8fc04f5b None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:25:30.754716 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-48c8900b-bb51-4ed4-b767-42dc8fc04f5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:25:30.854859 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:25:30.857329 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 506/1025] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:25:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 31 12:25:31.139067 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d469603c-9ced-4f14-aa16-cdd2a7e764d7 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:25:31.140634 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-d469603c-9ced-4f14-aa16-cdd2a7e764d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:25:31.246150 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d469603c-9ced-4f14-aa16-cdd2a7e764d7 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:25:31.299735 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:25:31.326649 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0213c145-d5e0-4c28-9ba3-60120a7fb80b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:25:31.327233 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0213c145-d5e0-4c28-9ba3-60120a7fb80b service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:25:31.327509 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0213c145-d5e0-4c28-9ba3-60120a7fb80b service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:25:31.327780 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0213c145-d5e0-4c28-9ba3-60120a7fb80b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:25:31.329963 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-0213c145-d5e0-4c28-9ba3-60120a7fb80b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:25:31.435772 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0213c145-d5e0-4c28-9ba3-60120a7fb80b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:25:31.437179 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0213c145-d5e0-4c28-9ba3-60120a7fb80b service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:25:31.455726 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 520/1026] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:25:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:25:36.401056 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e381cdd2-c529-4ecb-b425-54ff08362c52 None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:25:36.401958 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e381cdd2-c529-4ecb-b425-54ff08362c52 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:25:36.402441 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e381cdd2-c529-4ecb-b425-54ff08362c52 None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:25:36.404626 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 507/1027] 10.222.0.15 () {54 vars in 921 bytes} [Mon Oct 31 12:25:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:25:36.420953 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472b5bb6-100b-4be2-85d6-aac60395c495 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:25:36.421846 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472b5bb6-100b-4be2-85d6-aac60395c495 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:25:36.422313 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472b5bb6-100b-4be2-85d6-aac60395c495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:25:36.554549 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-472b5bb6-100b-4be2-85d6-aac60395c495 None None] MFA Rules not processed for user `9e7777495d7a406c8c50b1ad52e47263`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:25:36.555755 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-472b5bb6-100b-4be2-85d6-aac60395c495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:25:36.575505 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:25:36.675631 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 521/1028] 10.222.0.15 () {58 vars in 1018 bytes} [Mon Oct 31 12:25:36 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:25:40.877816 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36335687-5b6b-4152-9c7e-55f2eec4403e None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:25:40.878390 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36335687-5b6b-4152-9c7e-55f2eec4403e None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:25:40.878390 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36335687-5b6b-4152-9c7e-55f2eec4403e None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:25:40.880064 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 508/1029] 10.222.0.15 () {54 vars in 921 bytes} [Mon Oct 31 12:25:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:25:40.885145 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdf1261-117d-4982-8e19-ce9c3cbe577a None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:25:40.885615 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdf1261-117d-4982-8e19-ce9c3cbe577a None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:25:40.885920 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdf1261-117d-4982-8e19-ce9c3cbe577a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:25:40.966546 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-ccdf1261-117d-4982-8e19-ce9c3cbe577a None None] MFA Rules not processed for user `9e7777495d7a406c8c50b1ad52e47263`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:25:40.967933 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-ccdf1261-117d-4982-8e19-ce9c3cbe577a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:25:41.052311 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:25:41.100232 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 522/1030] 10.222.0.15 () {58 vars in 1018 bytes} [Mon Oct 31 12:25:40 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:25:42.356953 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b2ca33-1ea5-419f-8582-99acd96766fe None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:25:42.357461 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b2ca33-1ea5-419f-8582-99acd96766fe None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:25:42.357461 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b2ca33-1ea5-419f-8582-99acd96766fe None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:25:42.358771 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 509/1031] 10.222.0.15 () {54 vars in 921 bytes} [Mon Oct 31 12:25:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:25:42.364972 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8105f806-e238-4066-8e0e-2005a74261c0 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:25:42.365414 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8105f806-e238-4066-8e0e-2005a74261c0 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:25:42.365544 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8105f806-e238-4066-8e0e-2005a74261c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:25:42.445441 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-8105f806-e238-4066-8e0e-2005a74261c0 None None] MFA Rules not processed for user `9e7777495d7a406c8c50b1ad52e47263`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:25:42.446645 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-8105f806-e238-4066-8e0e-2005a74261c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:25:42.554321 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 523/1032] 10.222.0.15 () {58 vars in 1018 bytes} [Mon Oct 31 12:25:42 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:25:46.149529 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f750ecbe-5174-4e27-8929-7e579d9d0522 None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:25:46.150398 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f750ecbe-5174-4e27-8929-7e579d9d0522 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:25:46.150675 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f750ecbe-5174-4e27-8929-7e579d9d0522 None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:25:46.152140 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 510/1033] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:25:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:25:46.160909 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a361ee-3427-46d7-95d5-e7ca8eb48568 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:25:46.161388 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a361ee-3427-46d7-95d5-e7ca8eb48568 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:25:46.161578 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a361ee-3427-46d7-95d5-e7ca8eb48568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:25:46.200082 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:25:46.234377 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-75a361ee-3427-46d7-95d5-e7ca8eb48568 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:25:46.235907 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-75a361ee-3427-46d7-95d5-e7ca8eb48568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:25:46.344207 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 524/1034] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:25:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:25:46.643713 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a078247f-99b8-47e8-b8c8-71e90c297344 None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:25:46.644175 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a078247f-99b8-47e8-b8c8-71e90c297344 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:25:46.644592 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a078247f-99b8-47e8-b8c8-71e90c297344 None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:25:46.646503 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 511/1035] 10.222.0.15 () {54 vars in 961 bytes} [Mon Oct 31 12:25:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:25:46.673767 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673834c0-8883-48d5-a2ab-b6312c0ee16a None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:25:46.674422 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673834c0-8883-48d5-a2ab-b6312c0ee16a None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:25:46.674895 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673834c0-8883-48d5-a2ab-b6312c0ee16a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:25:46.768161 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-673834c0-8883-48d5-a2ab-b6312c0ee16a None None] MFA Rules not processed for user `99889b1dab0b471d9c77887cffefc652`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:25:46.769455 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-673834c0-8883-48d5-a2ab-b6312c0ee16a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:25:46.821692 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:25:46.893211 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 525/1036] 10.222.0.15 () {58 vars in 1058 bytes} [Mon Oct 31 12:25:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:25:46.900886 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a078247f-99b8-47e8-b8c8-71e90c297344 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:25:46.902367 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-a078247f-99b8-47e8-b8c8-71e90c297344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:25:47.060086 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a078247f-99b8-47e8-b8c8-71e90c297344 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:25:47.139736 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:25:47.184092 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c96545-5c49-4ea8-a300-d19d2c6e0b75 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99889b1dab0b471d9c77887cffefc652', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:25:47.184738 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c96545-5c49-4ea8-a300-d19d2c6e0b75 service nova] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:25:47.184957 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c96545-5c49-4ea8-a300-d19d2c6e0b75 service nova] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:25:47.185214 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c96545-5c49-4ea8-a300-d19d2c6e0b75 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:25:47.186835 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-d5c96545-5c49-4ea8-a300-d19d2c6e0b75 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:25:47.289638 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c96545-5c49-4ea8-a300-d19d2c6e0b75 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:25:47.291257 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c96545-5c49-4ea8-a300-d19d2c6e0b75 service nova] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:25:47.312058 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 512/1037] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:25:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:25:49.731510 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dd75d2-9b3a-4dd1-bf93-f0a0d1c35aac None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:25:49.732066 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dd75d2-9b3a-4dd1-bf93-f0a0d1c35aac None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:25:49.732271 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dd75d2-9b3a-4dd1-bf93-f0a0d1c35aac None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:25:49.733734 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 526/1038] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:25:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:25:49.739292 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2946db1f-0140-49d2-9543-ab009e137df6 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:25:49.739777 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2946db1f-0140-49d2-9543-ab009e137df6 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:25:49.739859 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2946db1f-0140-49d2-9543-ab009e137df6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:25:49.821310 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-2946db1f-0140-49d2-9543-ab009e137df6 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:25:49.822784 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-2946db1f-0140-49d2-9543-ab009e137df6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:25:49.864295 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:25:49.932855 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 513/1039] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:25:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:25:50.227287 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dd75d2-9b3a-4dd1-bf93-f0a0d1c35aac None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:25:50.228800 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-68dd75d2-9b3a-4dd1-bf93-f0a0d1c35aac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:25:50.423846 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dd75d2-9b3a-4dd1-bf93-f0a0d1c35aac None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:25:50.436131 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:25:50.586537 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9797b17-36ca-4f1b-9448-2ecf5be73612 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99889b1dab0b471d9c77887cffefc652', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:25:50.587152 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9797b17-36ca-4f1b-9448-2ecf5be73612 service nova] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:25:50.587487 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9797b17-36ca-4f1b-9448-2ecf5be73612 service nova] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:25:50.587759 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9797b17-36ca-4f1b-9448-2ecf5be73612 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:25:50.589336 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-b9797b17-36ca-4f1b-9448-2ecf5be73612 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:25:50.685010 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9797b17-36ca-4f1b-9448-2ecf5be73612 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:25:50.686595 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9797b17-36ca-4f1b-9448-2ecf5be73612 service nova] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:25:50.706610 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 527/1040] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:25:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:25:53.222204 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83528d3c-045f-4e36-815f-57adb9a3b568 None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:25:53.222739 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83528d3c-045f-4e36-815f-57adb9a3b568 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:25:53.222739 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83528d3c-045f-4e36-815f-57adb9a3b568 None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:25:53.224320 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 514/1041] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:25:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:25:53.229461 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106180fa-762f-420f-b6cb-24c82e0ab05c None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:25:53.229880 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106180fa-762f-420f-b6cb-24c82e0ab05c None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:25:53.230119 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106180fa-762f-420f-b6cb-24c82e0ab05c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:25:53.245917 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:25:53.301789 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-106180fa-762f-420f-b6cb-24c82e0ab05c None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:25:53.303578 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-106180fa-762f-420f-b6cb-24c82e0ab05c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:25:53.408711 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 528/1042] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:25:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:25:53.724848 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb0abc7-43d4-425b-9735-dda7c814ab82 None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:25:53.725361 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb0abc7-43d4-425b-9735-dda7c814ab82 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:25:53.725633 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb0abc7-43d4-425b-9735-dda7c814ab82 None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:25:53.727371 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 515/1043] 10.222.0.15 () {54 vars in 961 bytes} [Mon Oct 31 12:25:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:25:53.732822 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feae6b4c-d159-46e8-82e4-eea3f2ebef57 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:25:53.733204 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feae6b4c-d159-46e8-82e4-eea3f2ebef57 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:25:53.733571 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feae6b4c-d159-46e8-82e4-eea3f2ebef57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:25:53.815273 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-feae6b4c-d159-46e8-82e4-eea3f2ebef57 None None] MFA Rules not processed for user `99889b1dab0b471d9c77887cffefc652`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:25:53.816636 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-feae6b4c-d159-46e8-82e4-eea3f2ebef57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:25:53.853429 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:25:53.940658 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 529/1044] 10.222.0.15 () {58 vars in 1058 bytes} [Mon Oct 31 12:25:53 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:25:53.949699 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb0abc7-43d4-425b-9735-dda7c814ab82 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:25:53.951346 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-aeb0abc7-43d4-425b-9735-dda7c814ab82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:25:54.118333 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb0abc7-43d4-425b-9735-dda7c814ab82 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:25:54.139681 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:25:54.242389 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b04416b-45cb-400f-bfbc-05136290fd5c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99889b1dab0b471d9c77887cffefc652', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:25:54.243075 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b04416b-45cb-400f-bfbc-05136290fd5c service nova] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:25:54.243343 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b04416b-45cb-400f-bfbc-05136290fd5c service nova] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:25:54.243530 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b04416b-45cb-400f-bfbc-05136290fd5c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:25:54.245126 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-3b04416b-45cb-400f-bfbc-05136290fd5c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:25:54.354526 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b04416b-45cb-400f-bfbc-05136290fd5c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:25:54.356516 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b04416b-45cb-400f-bfbc-05136290fd5c service nova] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:25:54.379475 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 516/1045] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:25:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 31 12:25:54.383869 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9af9b48-7464-437e-b6fb-68b96828a98b None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:25:54.384318 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9af9b48-7464-437e-b6fb-68b96828a98b None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:25:54.384487 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9af9b48-7464-437e-b6fb-68b96828a98b None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:25:54.385747 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 530/1046] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:25:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:25:54.391443 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcb4f3c-0b18-466c-990b-86c0e2253f9d None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:25:54.392075 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcb4f3c-0b18-466c-990b-86c0e2253f9d None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:25:54.392415 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcb4f3c-0b18-466c-990b-86c0e2253f9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:25:54.426881 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:25:54.468370 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-3fcb4f3c-0b18-466c-990b-86c0e2253f9d None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:25:54.470101 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-3fcb4f3c-0b18-466c-990b-86c0e2253f9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:25:54.613121 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 517/1047] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:25:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:25:54.912740 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f623ae5-f104-4122-b793-cfe475a31dce None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:25:54.913027 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f623ae5-f104-4122-b793-cfe475a31dce None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:25:54.913297 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f623ae5-f104-4122-b793-cfe475a31dce None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:25:54.914769 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 531/1048] 10.222.0.15 () {54 vars in 963 bytes} [Mon Oct 31 12:25:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:25:54.919906 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d46ab8-8a45-4269-b946-a37eeda168ee None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:25:54.920258 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d46ab8-8a45-4269-b946-a37eeda168ee None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:25:54.920538 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d46ab8-8a45-4269-b946-a37eeda168ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:25:55.003620 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-c2d46ab8-8a45-4269-b946-a37eeda168ee None None] MFA Rules not processed for user `20d6d7ea077d49e8a66c6c6762499702`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:25:55.004876 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-c2d46ab8-8a45-4269-b946-a37eeda168ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:25:55.061130 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:25:55.099322 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 518/1049] 10.222.0.15 () {58 vars in 1060 bytes} [Mon Oct 31 12:25:54 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:25:55.106785 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f623ae5-f104-4122-b793-cfe475a31dce None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:25:55.108331 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-0f623ae5-f104-4122-b793-cfe475a31dce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:25:55.221937 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f623ae5-f104-4122-b793-cfe475a31dce None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:25:55.268675 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:25:55.308754 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7f9946-28cc-4639-9710-a0aa1e6ed6db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d6d7ea077d49e8a66c6c6762499702', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:25:55.309315 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7f9946-28cc-4639-9710-a0aa1e6ed6db service cinder] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:25:55.309556 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7f9946-28cc-4639-9710-a0aa1e6ed6db service cinder] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:25:55.309953 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7f9946-28cc-4639-9710-a0aa1e6ed6db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:25:55.311539 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-8a7f9946-28cc-4639-9710-a0aa1e6ed6db service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:25:55.444594 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7f9946-28cc-4639-9710-a0aa1e6ed6db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:25:55.446012 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7f9946-28cc-4639-9710-a0aa1e6ed6db service cinder] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:25:55.466691 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 532/1050] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:25:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:25:56.646241 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3206b374-ef08-44a4-8a2c-3d50ce8402f3 None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:25:56.646827 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3206b374-ef08-44a4-8a2c-3d50ce8402f3 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:25:56.646827 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3206b374-ef08-44a4-8a2c-3d50ce8402f3 None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:25:56.648204 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 519/1051] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:25:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:25:56.653474 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4a7866-653c-40d2-9c18-56f2eb8c7555 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:25:56.653899 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4a7866-653c-40d2-9c18-56f2eb8c7555 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:25:56.653993 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4a7866-653c-40d2-9c18-56f2eb8c7555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:25:56.721377 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-9b4a7866-653c-40d2-9c18-56f2eb8c7555 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:25:56.722608 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-9b4a7866-653c-40d2-9c18-56f2eb8c7555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:25:56.744150 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:25:56.821782 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 533/1052] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:25:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:25:57.110353 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3206b374-ef08-44a4-8a2c-3d50ce8402f3 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:25:57.111902 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-3206b374-ef08-44a4-8a2c-3d50ce8402f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:25:57.265232 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3206b374-ef08-44a4-8a2c-3d50ce8402f3 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:25:57.320093 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:25:57.386406 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5644de1-63ba-4cee-83bf-ad64c31c450b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99889b1dab0b471d9c77887cffefc652', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:25:57.387048 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5644de1-63ba-4cee-83bf-ad64c31c450b service nova] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:25:57.387360 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5644de1-63ba-4cee-83bf-ad64c31c450b service nova] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:25:57.387630 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5644de1-63ba-4cee-83bf-ad64c31c450b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:25:57.389222 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-a5644de1-63ba-4cee-83bf-ad64c31c450b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:25:57.488777 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5644de1-63ba-4cee-83bf-ad64c31c450b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:25:57.490241 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5644de1-63ba-4cee-83bf-ad64c31c450b service nova] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:25:57.510743 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 520/1053] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:25:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:25:59.809113 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7896966-0ceb-41e5-9d5a-e0decf6e4b16 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:25:59.809680 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7896966-0ceb-41e5-9d5a-e0decf6e4b16 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:25:59.809680 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7896966-0ceb-41e5-9d5a-e0decf6e4b16 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:25:59.810983 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 534/1054] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:25:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:25:59.817567 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689c0d84-d98a-4e3d-a552-e1f211d4d897 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:25:59.818110 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689c0d84-d98a-4e3d-a552-e1f211d4d897 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:25:59.818110 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689c0d84-d98a-4e3d-a552-e1f211d4d897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:25:59.889523 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-689c0d84-d98a-4e3d-a552-e1f211d4d897 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:25:59.891338 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-689c0d84-d98a-4e3d-a552-e1f211d4d897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:25:59.898907 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:25:59.994125 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 521/1055] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:25:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:26:00.287635 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7896966-0ceb-41e5-9d5a-e0decf6e4b16 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:26:00.289136 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-a7896966-0ceb-41e5-9d5a-e0decf6e4b16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:26:00.441475 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7896966-0ceb-41e5-9d5a-e0decf6e4b16 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:26:00.445225 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:26:00.559979 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad3dd65-01f5-4149-94f7-3da0e4efbe1d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99889b1dab0b471d9c77887cffefc652', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:26:00.560506 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad3dd65-01f5-4149-94f7-3da0e4efbe1d service nova] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:26:00.560786 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad3dd65-01f5-4149-94f7-3da0e4efbe1d service nova] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:26:00.561059 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad3dd65-01f5-4149-94f7-3da0e4efbe1d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:26:00.563225 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-8ad3dd65-01f5-4149-94f7-3da0e4efbe1d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:26:00.662978 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad3dd65-01f5-4149-94f7-3da0e4efbe1d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:26:00.664593 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad3dd65-01f5-4149-94f7-3da0e4efbe1d service nova] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:26:00.684196 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 535/1056] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:26:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:26:02.919860 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc0ae50-41de-4691-bae6-5d549cf23fce None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:26:02.920299 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc0ae50-41de-4691-bae6-5d549cf23fce None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:26:02.920469 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc0ae50-41de-4691-bae6-5d549cf23fce None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:26:02.921822 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 522/1057] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:26:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:26:02.927276 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c93340b-94a2-46da-805e-e797779c5c15 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:26:02.927663 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c93340b-94a2-46da-805e-e797779c5c15 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:26:02.927824 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c93340b-94a2-46da-805e-e797779c5c15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:26:02.940173 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:26:02.998982 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-7c93340b-94a2-46da-805e-e797779c5c15 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:26:03.000574 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-7c93340b-94a2-46da-805e-e797779c5c15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:26:03.112879 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 536/1058] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:26:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:26:03.408007 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc0ae50-41de-4691-bae6-5d549cf23fce None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:26:03.409507 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-8fc0ae50-41de-4691-bae6-5d549cf23fce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:26:03.582003 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:26:03.597177 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc0ae50-41de-4691-bae6-5d549cf23fce None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:26:03.755642 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a414c211-8df8-460d-a424-ec1bb1fbdb20 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99889b1dab0b471d9c77887cffefc652', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:26:03.756342 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a414c211-8df8-460d-a424-ec1bb1fbdb20 service nova] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:26:03.756751 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a414c211-8df8-460d-a424-ec1bb1fbdb20 service nova] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:26:03.757036 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a414c211-8df8-460d-a424-ec1bb1fbdb20 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:26:03.759260 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-a414c211-8df8-460d-a424-ec1bb1fbdb20 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:26:03.865770 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a414c211-8df8-460d-a424-ec1bb1fbdb20 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:26:03.867488 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a414c211-8df8-460d-a424-ec1bb1fbdb20 service nova] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:26:03.878148 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:26:03.887080 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 523/1059] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:26:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:26:06.750927 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01591a02-18ae-49cd-b91f-6ed763cadc07 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:26:06.752181 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01591a02-18ae-49cd-b91f-6ed763cadc07 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:26:06.752181 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01591a02-18ae-49cd-b91f-6ed763cadc07 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:26:06.753564 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 537/1060] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:26:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:26:06.759670 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146a238c-6e82-46d2-95e9-d74017a72276 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:26:06.760158 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146a238c-6e82-46d2-95e9-d74017a72276 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:26:06.760376 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146a238c-6e82-46d2-95e9-d74017a72276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:26:06.844220 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-146a238c-6e82-46d2-95e9-d74017a72276 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:26:06.845469 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-146a238c-6e82-46d2-95e9-d74017a72276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:26:06.943824 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 524/1061] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:26:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:26:07.233496 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01591a02-18ae-49cd-b91f-6ed763cadc07 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:26:07.235594 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-01591a02-18ae-49cd-b91f-6ed763cadc07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:26:07.331323 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:26:07.394583 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01591a02-18ae-49cd-b91f-6ed763cadc07 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:26:07.527537 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e418b2c-db54-4856-aed3-9890f07d0f91 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99889b1dab0b471d9c77887cffefc652', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:26:07.528170 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e418b2c-db54-4856-aed3-9890f07d0f91 service nova] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:26:07.528428 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e418b2c-db54-4856-aed3-9890f07d0f91 service nova] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:26:07.528701 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e418b2c-db54-4856-aed3-9890f07d0f91 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:26:07.530281 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-2e418b2c-db54-4856-aed3-9890f07d0f91 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:26:07.582031 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:26:07.624716 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e418b2c-db54-4856-aed3-9890f07d0f91 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:26:07.626305 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e418b2c-db54-4856-aed3-9890f07d0f91 service nova] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:26:07.646575 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 538/1062] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:26:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:26:09.805409 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d027ca-bdd0-4bc2-93be-df677c888019 None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:26:09.806092 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d027ca-bdd0-4bc2-93be-df677c888019 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:26:09.806092 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d027ca-bdd0-4bc2-93be-df677c888019 None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:26:09.807322 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 525/1063] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:26:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:26:09.812714 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61affa1e-4ac1-496f-b457-01b46eda4a8d None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:26:09.813122 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61affa1e-4ac1-496f-b457-01b46eda4a8d None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:26:09.813258 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61affa1e-4ac1-496f-b457-01b46eda4a8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:26:09.880520 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-61affa1e-4ac1-496f-b457-01b46eda4a8d None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:26:09.881719 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-61affa1e-4ac1-496f-b457-01b46eda4a8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:26:09.984240 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 539/1064] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:26:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:26:10.273452 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d027ca-bdd0-4bc2-93be-df677c888019 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:26:10.274955 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-23d027ca-bdd0-4bc2-93be-df677c888019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:26:10.353613 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:26:10.428077 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d027ca-bdd0-4bc2-93be-df677c888019 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:26:10.543103 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dce13eb-fbf5-459c-82c5-906301840bfd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99889b1dab0b471d9c77887cffefc652', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:26:10.543736 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dce13eb-fbf5-459c-82c5-906301840bfd service nova] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:26:10.544011 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dce13eb-fbf5-459c-82c5-906301840bfd service nova] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:26:10.544330 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dce13eb-fbf5-459c-82c5-906301840bfd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:26:10.545891 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-7dce13eb-fbf5-459c-82c5-906301840bfd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:26:10.585376 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:26:10.638663 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dce13eb-fbf5-459c-82c5-906301840bfd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:26:10.640290 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dce13eb-fbf5-459c-82c5-906301840bfd service nova] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:26:10.660476 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 526/1065] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:26:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:26:10.794836 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a570759-0192-4c96-bf2d-8c77bc3caf18 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:26:10.795052 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a570759-0192-4c96-bf2d-8c77bc3caf18 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:26:10.795337 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a570759-0192-4c96-bf2d-8c77bc3caf18 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:26:10.796737 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 540/1066] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:26:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:26:10.803198 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065c745f-740a-4378-b8d4-f21f45b325fb None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:26:10.803490 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065c745f-740a-4378-b8d4-f21f45b325fb None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:26:10.803762 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065c745f-740a-4378-b8d4-f21f45b325fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:26:10.876895 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-065c745f-740a-4378-b8d4-f21f45b325fb None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:26:10.878235 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-065c745f-740a-4378-b8d4-f21f45b325fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:26:10.981616 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 527/1067] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:26:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:26:11.285267 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3edb5f-965a-4208-a9de-0d55df794e03 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:26:11.285754 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3edb5f-965a-4208-a9de-0d55df794e03 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:26:11.285854 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3edb5f-965a-4208-a9de-0d55df794e03 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:26:11.287117 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 541/1068] 10.222.0.15 () {54 vars in 962 bytes} [Mon Oct 31 12:26:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:26:11.292373 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2689731-80c6-4566-8ce9-fb5a7f581169 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:26:11.292655 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2689731-80c6-4566-8ce9-fb5a7f581169 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:26:11.292922 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2689731-80c6-4566-8ce9-fb5a7f581169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:26:11.309717 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:26:11.373151 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-c2689731-80c6-4566-8ce9-fb5a7f581169 None None] MFA Rules not processed for user `4feaa01c5ed64b65b10d4dece823702c`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:26:11.374302 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-c2689731-80c6-4566-8ce9-fb5a7f581169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:26:11.451927 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 528/1069] 10.222.0.15 () {58 vars in 1059 bytes} [Mon Oct 31 12:26:11 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 31 12:26:11.459897 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3edb5f-965a-4208-a9de-0d55df794e03 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:26:11.461380 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-6b3edb5f-965a-4208-a9de-0d55df794e03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:26:11.493095 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:26:11.566424 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3edb5f-965a-4208-a9de-0d55df794e03 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:26:11.645770 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0401c8eb-5077-4ddc-a90b-be304049e668 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4feaa01c5ed64b65b10d4dece823702c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:26:11.646304 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0401c8eb-5077-4ddc-a90b-be304049e668 service glance] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:26:11.646566 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0401c8eb-5077-4ddc-a90b-be304049e668 service glance] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:26:11.646831 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0401c8eb-5077-4ddc-a90b-be304049e668 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:26:11.648910 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-0401c8eb-5077-4ddc-a90b-be304049e668 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:26:11.727288 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:26:11.759339 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0401c8eb-5077-4ddc-a90b-be304049e668 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:26:11.760780 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0401c8eb-5077-4ddc-a90b-be304049e668 service glance] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:26:11.778914 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 542/1070] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:26:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:26:12.535102 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa28150-0b4c-4cd5-a7b1-6b376797df9e None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:26:12.535657 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa28150-0b4c-4cd5-a7b1-6b376797df9e None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:26:12.535657 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa28150-0b4c-4cd5-a7b1-6b376797df9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:26:12.602727 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-5aa28150-0b4c-4cd5-a7b1-6b376797df9e None None] MFA Rules not processed for user `bf4faacf7df248d28f32d7014dadcc07`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:26:12.603970 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-5aa28150-0b4c-4cd5-a7b1-6b376797df9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:26:12.659892 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:26:12.693211 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 529/1071] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Oct 31 12:26:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:26:12.738623 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1023350f-2b1a-4469-b85c-24a9cb96c7a7 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:26:12.739211 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1023350f-2b1a-4469-b85c-24a9cb96c7a7 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:26:12.739211 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1023350f-2b1a-4469-b85c-24a9cb96c7a7 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:26:12.740632 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 543/1072] 10.222.0.15 () {54 vars in 943 bytes} [Mon Oct 31 12:26:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:26:12.747817 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a22e6-28c7-40da-b938-e829150fddac None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:26:12.748105 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a22e6-28c7-40da-b938-e829150fddac None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:26:12.748378 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a22e6-28c7-40da-b938-e829150fddac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:26:12.830862 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-dd6a22e6-28c7-40da-b938-e829150fddac None None] MFA Rules not processed for user `b2254a5786c249b4aa79244f7198ae65`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:26:12.832191 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-dd6a22e6-28c7-40da-b938-e829150fddac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:26:12.859486 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c749bff4-7e24-49b2-912d-c154dc833cb6 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:26:12.859767 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c749bff4-7e24-49b2-912d-c154dc833cb6 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:26:12.860034 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c749bff4-7e24-49b2-912d-c154dc833cb6 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:26:12.862039 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 544/1073] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:26:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 31 12:26:12.866700 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef0464b-dbc4-46bf-b17f-d1bac414b618 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:26:12.867001 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef0464b-dbc4-46bf-b17f-d1bac414b618 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:26:12.867275 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef0464b-dbc4-46bf-b17f-d1bac414b618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:26:12.937089 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-7ef0464b-dbc4-46bf-b17f-d1bac414b618 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:26:12.938294 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-7ef0464b-dbc4-46bf-b17f-d1bac414b618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:26:12.945511 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 530/1074] 10.222.0.15 () {58 vars in 1040 bytes} [Mon Oct 31 12:26:12 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:26:12.954809 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6a22e6-28c7-40da-b938-e829150fddac None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:26:12.964312 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6a22e6-28c7-40da-b938-e829150fddac None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:26:12.991029 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:26:13.040139 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 545/1075] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:26:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 31 12:26:13.073101 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18f8c9f-9cf4-491a-a877-12c2d912bea0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2254a5786c249b4aa79244f7198ae65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:26:13.073699 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18f8c9f-9cf4-491a-a877-12c2d912bea0 service swift] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:26:13.073971 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18f8c9f-9cf4-491a-a877-12c2d912bea0 service swift] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:26:13.074235 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18f8c9f-9cf4-491a-a877-12c2d912bea0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:26:13.075972 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-f18f8c9f-9cf4-491a-a877-12c2d912bea0 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:26:13.159309 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18f8c9f-9cf4-491a-a877-12c2d912bea0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:26:13.160884 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18f8c9f-9cf4-491a-a877-12c2d912bea0 service swift] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:26:13.168880 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 531/1076] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Oct 31 12:26:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 31 12:26:13.327537 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef0464b-dbc4-46bf-b17f-d1bac414b618 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:26:13.329186 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-7ef0464b-dbc4-46bf-b17f-d1bac414b618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:26:13.387291 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:26:13.470928 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef0464b-dbc4-46bf-b17f-d1bac414b618 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:26:13.600431 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c609d5-a687-4835-aecd-f0ff072d9677 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99889b1dab0b471d9c77887cffefc652', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:26:13.601251 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c609d5-a687-4835-aecd-f0ff072d9677 service nova] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:26:13.602373 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c609d5-a687-4835-aecd-f0ff072d9677 service nova] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:26:13.602674 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c609d5-a687-4835-aecd-f0ff072d9677 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:26:13.609095 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-d5c609d5-a687-4835-aecd-f0ff072d9677 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:26:13.656052 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:26:13.716074 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c609d5-a687-4835-aecd-f0ff072d9677 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:26:13.717688 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c609d5-a687-4835-aecd-f0ff072d9677 service nova] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:26:13.743397 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 546/1077] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:26:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:26:15.963887 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3bead3-d498-409e-be14-2194253659da None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:26:15.964480 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3bead3-d498-409e-be14-2194253659da None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:26:15.964480 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3bead3-d498-409e-be14-2194253659da None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:26:15.965928 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 532/1078] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:26:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:26:15.971248 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240dac27-0905-4fb2-8b82-dd6cc539e463 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:26:15.971672 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240dac27-0905-4fb2-8b82-dd6cc539e463 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:26:15.971827 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240dac27-0905-4fb2-8b82-dd6cc539e463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:26:16.044946 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-240dac27-0905-4fb2-8b82-dd6cc539e463 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:26:16.046758 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-240dac27-0905-4fb2-8b82-dd6cc539e463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:26:16.163302 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 547/1079] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:26:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:26:16.451201 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3bead3-d498-409e-be14-2194253659da None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:26:16.453047 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-2a3bead3-d498-409e-be14-2194253659da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:26:16.520407 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:26:16.620505 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3bead3-d498-409e-be14-2194253659da None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:26:16.763912 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78f328e-4d97-41de-a267-2e6451122367 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99889b1dab0b471d9c77887cffefc652', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:26:16.764426 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78f328e-4d97-41de-a267-2e6451122367 service nova] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:26:16.764700 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78f328e-4d97-41de-a267-2e6451122367 service nova] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:26:16.764968 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78f328e-4d97-41de-a267-2e6451122367 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:26:16.767353 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-d78f328e-4d97-41de-a267-2e6451122367 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:26:16.790308 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:26:16.873433 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78f328e-4d97-41de-a267-2e6451122367 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:26:16.875376 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78f328e-4d97-41de-a267-2e6451122367 service nova] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:26:16.897700 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 533/1080] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:26:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:26:19.139077 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f72b99f-fba7-4164-b1bb-b1dbd504c6b7 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:26:19.139723 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f72b99f-fba7-4164-b1bb-b1dbd504c6b7 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:26:19.139723 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f72b99f-fba7-4164-b1bb-b1dbd504c6b7 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:26:19.141000 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 548/1081] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:26:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:26:19.146989 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef1858-9d6e-406b-800a-8733db29b8e0 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:26:19.147456 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef1858-9d6e-406b-800a-8733db29b8e0 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:26:19.147549 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef1858-9d6e-406b-800a-8733db29b8e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:26:19.218918 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-daef1858-9d6e-406b-800a-8733db29b8e0 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:26:19.220646 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-daef1858-9d6e-406b-800a-8733db29b8e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:26:19.328187 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:26:19.336129 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 534/1082] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:26:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:26:19.629537 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f72b99f-fba7-4164-b1bb-b1dbd504c6b7 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:26:19.631051 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-0f72b99f-fba7-4164-b1bb-b1dbd504c6b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:26:19.643300 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:26:19.795854 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f72b99f-fba7-4164-b1bb-b1dbd504c6b7 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:26:19.882016 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:26:19.919189 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18090a35-1f27-49d2-b486-bc287c5dd5dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99889b1dab0b471d9c77887cffefc652', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:26:19.919800 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18090a35-1f27-49d2-b486-bc287c5dd5dc service nova] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:26:19.920074 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18090a35-1f27-49d2-b486-bc287c5dd5dc service nova] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:26:19.920336 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18090a35-1f27-49d2-b486-bc287c5dd5dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:26:19.922063 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-18090a35-1f27-49d2-b486-bc287c5dd5dc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:26:20.022989 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18090a35-1f27-49d2-b486-bc287c5dd5dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:26:20.024600 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18090a35-1f27-49d2-b486-bc287c5dd5dc service nova] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:26:20.044089 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 549/1083] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:26:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:26:22.290527 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb7f291-6ba5-4697-a054-67b1b094a5d8 None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:26:22.291043 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb7f291-6ba5-4697-a054-67b1b094a5d8 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:26:22.291043 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb7f291-6ba5-4697-a054-67b1b094a5d8 None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:26:22.296578 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 535/1084] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:26:22 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:26:22.298975 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c4840d-dad1-4858-a435-768018784d29 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:26:22.299441 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c4840d-dad1-4858-a435-768018784d29 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:26:22.299441 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c4840d-dad1-4858-a435-768018784d29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:26:22.378942 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-20c4840d-dad1-4858-a435-768018784d29 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:26:22.380206 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-20c4840d-dad1-4858-a435-768018784d29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:26:22.435372 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:26:22.498234 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 550/1085] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:26:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 31 12:26:22.801856 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb7f291-6ba5-4697-a054-67b1b094a5d8 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:26:22.803414 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-adb7f291-6ba5-4697-a054-67b1b094a5d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:26:23.008379 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb7f291-6ba5-4697-a054-67b1b094a5d8 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:26:23.154672 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:26:23.241130 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf07a65-9f82-4a31-8c84-543b7ed737a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99889b1dab0b471d9c77887cffefc652', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:26:23.241786 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf07a65-9f82-4a31-8c84-543b7ed737a8 service nova] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:26:23.242061 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf07a65-9f82-4a31-8c84-543b7ed737a8 service nova] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:26:23.242329 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf07a65-9f82-4a31-8c84-543b7ed737a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:26:23.243984 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-8bf07a65-9f82-4a31-8c84-543b7ed737a8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:26:23.346741 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf07a65-9f82-4a31-8c84-543b7ed737a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:26:23.352620 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf07a65-9f82-4a31-8c84-543b7ed737a8 service nova] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:26:23.373546 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 536/1086] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:26:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 572 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:26:32.490001 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=23 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:26:33.370517 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=64 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:26:36.011014 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec9f1d1-e72e-47f7-b847-2662bd631ccc None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:26:36.011627 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec9f1d1-e72e-47f7-b847-2662bd631ccc None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:26:36.011627 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec9f1d1-e72e-47f7-b847-2662bd631ccc None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:26:36.012967 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 551/1087] 10.222.0.15 () {54 vars in 918 bytes} [Mon Oct 31 12:26:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:26:36.019081 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea3a307-701c-4402-88cd-3528e3291f89 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:26:36.019547 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea3a307-701c-4402-88cd-3528e3291f89 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:26:36.019663 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea3a307-701c-4402-88cd-3528e3291f89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:26:36.094549 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-8ea3a307-701c-4402-88cd-3528e3291f89 None None] MFA Rules not processed for user `9e7777495d7a406c8c50b1ad52e47263`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:26:36.095769 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-8ea3a307-701c-4402-88cd-3528e3291f89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:26:36.201029 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 537/1088] 10.222.0.15 () {58 vars in 1015 bytes} [Mon Oct 31 12:26:36 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:26:36.215658 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c7049a-caa7-4775-b3c4-b96311a60580 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:26:36.215940 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c7049a-caa7-4775-b3c4-b96311a60580 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:26:36.216212 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c7049a-caa7-4775-b3c4-b96311a60580 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:26:36.217540 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 552/1089] 10.222.0.15 () {54 vars in 918 bytes} [Mon Oct 31 12:26:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:26:36.222741 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3862d87e-6c90-43e2-8ab7-28c28b9160a2 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:26:36.223027 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3862d87e-6c90-43e2-8ab7-28c28b9160a2 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:26:36.223338 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3862d87e-6c90-43e2-8ab7-28c28b9160a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:26:36.295551 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:26:36.298824 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-3862d87e-6c90-43e2-8ab7-28c28b9160a2 None None] MFA Rules not processed for user `99889b1dab0b471d9c77887cffefc652`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:26:36.300560 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-3862d87e-6c90-43e2-8ab7-28c28b9160a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:26:36.413285 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 538/1090] 10.222.0.15 () {58 vars in 1015 bytes} [Mon Oct 31 12:26:36 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:26:38.974877 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2790fa4-13d7-41df-93de-8272ae325419 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:26:38.975481 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2790fa4-13d7-41df-93de-8272ae325419 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:26:38.975481 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2790fa4-13d7-41df-93de-8272ae325419 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:26:38.976822 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 553/1091] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 31 12:26:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:26:38.982193 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b6ad01-1871-4718-a111-1ad0361d0d0f None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:26:38.982613 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b6ad01-1871-4718-a111-1ad0361d0d0f None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:26:38.982736 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b6ad01-1871-4718-a111-1ad0361d0d0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:26:39.052071 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-d9b6ad01-1871-4718-a111-1ad0361d0d0f None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:26:39.053266 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-d9b6ad01-1871-4718-a111-1ad0361d0d0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:26:39.107582 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:26:39.149958 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 539/1092] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 31 12:26:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:26:39.442002 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2790fa4-13d7-41df-93de-8272ae325419 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:26:39.443697 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-d2790fa4-13d7-41df-93de-8272ae325419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:26:39.588503 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2790fa4-13d7-41df-93de-8272ae325419 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:26:39.660611 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:26:39.701357 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5629c77d-bc05-484b-a8da-35056d947adc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99889b1dab0b471d9c77887cffefc652', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:26:39.701887 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5629c77d-bc05-484b-a8da-35056d947adc service nova] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:26:39.702163 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5629c77d-bc05-484b-a8da-35056d947adc service nova] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:26:39.702432 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5629c77d-bc05-484b-a8da-35056d947adc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:26:39.704685 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-5629c77d-bc05-484b-a8da-35056d947adc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:26:39.792968 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5629c77d-bc05-484b-a8da-35056d947adc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:26:39.794542 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5629c77d-bc05-484b-a8da-35056d947adc service nova] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:26:39.813098 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 554/1093] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:26:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:26:40.932997 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb267b0b-4c1c-46d2-a85b-4f9c1e8a730b None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:26:40.933585 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb267b0b-4c1c-46d2-a85b-4f9c1e8a730b None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:26:40.933585 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb267b0b-4c1c-46d2-a85b-4f9c1e8a730b None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:26:40.934922 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 540/1094] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 31 12:26:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:26:40.940283 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c004d27b-3ef2-437e-8dcf-1e383b6f5814 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:26:40.940732 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c004d27b-3ef2-437e-8dcf-1e383b6f5814 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:26:40.940842 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c004d27b-3ef2-437e-8dcf-1e383b6f5814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:26:41.016734 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-c004d27b-3ef2-437e-8dcf-1e383b6f5814 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:26:41.017949 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-c004d27b-3ef2-437e-8dcf-1e383b6f5814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:26:41.044711 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:26:41.113331 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 555/1095] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 31 12:26:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:26:41.273972 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb267b0b-4c1c-46d2-a85b-4f9c1e8a730b None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:26:41.275493 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-cb267b0b-4c1c-46d2-a85b-4f9c1e8a730b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:26:41.417015 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb267b0b-4c1c-46d2-a85b-4f9c1e8a730b None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:26:41.457687 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:26:41.529633 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d50dfa1-aefb-48b6-bc67-a90861d4ee0a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99889b1dab0b471d9c77887cffefc652', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:26:41.530245 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d50dfa1-aefb-48b6-bc67-a90861d4ee0a service nova] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:26:41.530520 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d50dfa1-aefb-48b6-bc67-a90861d4ee0a service nova] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:26:41.530785 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d50dfa1-aefb-48b6-bc67-a90861d4ee0a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:26:41.532625 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-6d50dfa1-aefb-48b6-bc67-a90861d4ee0a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:26:41.619349 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d50dfa1-aefb-48b6-bc67-a90861d4ee0a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:26:41.620908 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d50dfa1-aefb-48b6-bc67-a90861d4ee0a service nova] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:26:41.638755 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 541/1096] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:26:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:26:42.751600 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd053bf5-4c82-4c95-bc09-f63d9d7207bc None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:26:42.752142 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd053bf5-4c82-4c95-bc09-f63d9d7207bc None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:26:42.752142 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd053bf5-4c82-4c95-bc09-f63d9d7207bc None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:26:42.753399 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 556/1097] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 31 12:26:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:26:42.758132 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d11f60b-cc7c-43c7-b866-25593b2bc719 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:26:42.758530 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d11f60b-cc7c-43c7-b866-25593b2bc719 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:26:42.758663 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d11f60b-cc7c-43c7-b866-25593b2bc719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:26:42.815492 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:26:42.837006 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-0d11f60b-cc7c-43c7-b866-25593b2bc719 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:26:42.838194 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-0d11f60b-cc7c-43c7-b866-25593b2bc719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:26:42.934926 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 542/1098] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 31 12:26:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:26:43.084317 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd053bf5-4c82-4c95-bc09-f63d9d7207bc None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:26:43.085831 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-fd053bf5-4c82-4c95-bc09-f63d9d7207bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:26:43.230516 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd053bf5-4c82-4c95-bc09-f63d9d7207bc None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:26:43.242756 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:26:43.343932 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdfa32a-549c-4a5d-8070-61ab9e00521d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99889b1dab0b471d9c77887cffefc652', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:26:43.344453 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdfa32a-549c-4a5d-8070-61ab9e00521d service nova] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:26:43.344728 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdfa32a-549c-4a5d-8070-61ab9e00521d service nova] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:26:43.344997 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdfa32a-549c-4a5d-8070-61ab9e00521d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:26:43.346594 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-7bdfa32a-549c-4a5d-8070-61ab9e00521d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:26:43.433338 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdfa32a-549c-4a5d-8070-61ab9e00521d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:26:43.434913 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdfa32a-549c-4a5d-8070-61ab9e00521d service nova] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:26:43.453928 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 557/1099] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:26:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:26:44.542732 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebc1617-04fd-4c35-9cf9-3111b624cd81 None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:26:44.543294 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebc1617-04fd-4c35-9cf9-3111b624cd81 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:26:44.543294 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebc1617-04fd-4c35-9cf9-3111b624cd81 None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:26:44.544644 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 543/1100] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 31 12:26:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:26:44.549947 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56d92d3-2da1-49bb-b242-fa3c0bad064a None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:26:44.550395 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56d92d3-2da1-49bb-b242-fa3c0bad064a None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:26:44.550492 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56d92d3-2da1-49bb-b242-fa3c0bad064a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:26:44.570542 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:26:44.627384 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-f56d92d3-2da1-49bb-b242-fa3c0bad064a None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:26:44.628583 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-f56d92d3-2da1-49bb-b242-fa3c0bad064a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:26:44.724675 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 558/1101] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 31 12:26:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:26:44.873635 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebc1617-04fd-4c35-9cf9-3111b624cd81 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:26:44.875597 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-aebc1617-04fd-4c35-9cf9-3111b624cd81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:26:44.984403 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:26:45.021177 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebc1617-04fd-4c35-9cf9-3111b624cd81 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:26:45.135140 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d436c61e-bff5-4ebe-8c68-0cb4183290f4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99889b1dab0b471d9c77887cffefc652', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:26:45.135769 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d436c61e-bff5-4ebe-8c68-0cb4183290f4 service nova] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:26:45.136068 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d436c61e-bff5-4ebe-8c68-0cb4183290f4 service nova] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:26:45.136334 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d436c61e-bff5-4ebe-8c68-0cb4183290f4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:26:45.137919 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-d436c61e-bff5-4ebe-8c68-0cb4183290f4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:26:45.212141 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:26:45.225970 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d436c61e-bff5-4ebe-8c68-0cb4183290f4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:26:45.227567 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d436c61e-bff5-4ebe-8c68-0cb4183290f4 service nova] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:26:45.245459 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 544/1102] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:26:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:26:46.347061 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc938dcf-93c0-401d-a051-a4eafc1f796b None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:26:46.347673 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc938dcf-93c0-401d-a051-a4eafc1f796b None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:26:46.347673 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc938dcf-93c0-401d-a051-a4eafc1f796b None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:26:46.348876 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 559/1103] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 31 12:26:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:26:46.353859 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f14daf-5be0-4d92-a147-a6114a7bd263 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:26:46.354286 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f14daf-5be0-4d92-a147-a6114a7bd263 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:26:46.354443 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f14daf-5be0-4d92-a147-a6114a7bd263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:26:46.430548 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-41f14daf-5be0-4d92-a147-a6114a7bd263 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:26:46.431790 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-41f14daf-5be0-4d92-a147-a6114a7bd263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:26:46.527845 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 545/1104] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 31 12:26:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:26:46.679807 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc938dcf-93c0-401d-a051-a4eafc1f796b None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:26:46.681235 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-dc938dcf-93c0-401d-a051-a4eafc1f796b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:26:46.766590 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:26:46.826855 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc938dcf-93c0-401d-a051-a4eafc1f796b None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:26:46.937599 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3251f314-3e87-4f7c-ba6a-03b74e0eeaf3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99889b1dab0b471d9c77887cffefc652', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:26:46.938253 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3251f314-3e87-4f7c-ba6a-03b74e0eeaf3 service nova] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:26:46.938253 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3251f314-3e87-4f7c-ba6a-03b74e0eeaf3 service nova] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:26:46.938445 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3251f314-3e87-4f7c-ba6a-03b74e0eeaf3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:26:46.939992 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-3251f314-3e87-4f7c-ba6a-03b74e0eeaf3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:26:46.987091 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:26:47.028936 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3251f314-3e87-4f7c-ba6a-03b74e0eeaf3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:26:47.030521 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3251f314-3e87-4f7c-ba6a-03b74e0eeaf3 service nova] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:26:47.048795 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 560/1105] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:26:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:26:48.084878 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5bad72-916f-455d-b66d-b427e33b980c None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:26:48.085417 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5bad72-916f-455d-b66d-b427e33b980c None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:26:48.085417 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5bad72-916f-455d-b66d-b427e33b980c None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:26:48.086822 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 546/1106] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 31 12:26:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 31 12:26:48.091575 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b912e67-c683-4fa0-9d18-29ff0e7b8079 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:26:48.091986 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b912e67-c683-4fa0-9d18-29ff0e7b8079 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:26:48.092115 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b912e67-c683-4fa0-9d18-29ff0e7b8079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:26:48.167856 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-2b912e67-c683-4fa0-9d18-29ff0e7b8079 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:26:48.169038 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-2b912e67-c683-4fa0-9d18-29ff0e7b8079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:26:48.264388 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 561/1107] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 31 12:26:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:26:48.413958 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5bad72-916f-455d-b66d-b427e33b980c None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:26:48.415575 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-ae5bad72-916f-455d-b66d-b427e33b980c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:26:48.464053 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:26:48.559908 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5bad72-916f-455d-b66d-b427e33b980c None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:26:48.673141 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff283f9-51ac-4a0e-8805-138e7df4ffb9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99889b1dab0b471d9c77887cffefc652', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:26:48.673745 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff283f9-51ac-4a0e-8805-138e7df4ffb9 service nova] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:26:48.674020 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff283f9-51ac-4a0e-8805-138e7df4ffb9 service nova] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:26:48.674292 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff283f9-51ac-4a0e-8805-138e7df4ffb9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:26:48.675894 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-9ff283f9-51ac-4a0e-8805-138e7df4ffb9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:26:48.682769 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:26:48.765256 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff283f9-51ac-4a0e-8805-138e7df4ffb9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:26:48.767076 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff283f9-51ac-4a0e-8805-138e7df4ffb9 service nova] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:26:48.785088 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 547/1108] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:26:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:26:49.820286 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ab70e2-aa20-4f3d-b7ea-61f929dfa4b2 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:26:49.820872 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ab70e2-aa20-4f3d-b7ea-61f929dfa4b2 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:26:49.820872 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ab70e2-aa20-4f3d-b7ea-61f929dfa4b2 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:26:49.822159 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 562/1109] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 31 12:26:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:26:49.826960 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9df1df-b51d-44ce-8ee2-0fc522d3c674 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:26:49.827881 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9df1df-b51d-44ce-8ee2-0fc522d3c674 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:26:49.828152 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9df1df-b51d-44ce-8ee2-0fc522d3c674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:26:49.903465 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-aa9df1df-b51d-44ce-8ee2-0fc522d3c674 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:26:49.904654 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-aa9df1df-b51d-44ce-8ee2-0fc522d3c674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:26:49.972376 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:26:50.001067 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 548/1110] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 31 12:26:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:26:50.148676 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ab70e2-aa20-4f3d-b7ea-61f929dfa4b2 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:26:50.150095 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-22ab70e2-aa20-4f3d-b7ea-61f929dfa4b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:26:50.168333 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:26:50.295505 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ab70e2-aa20-4f3d-b7ea-61f929dfa4b2 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:26:50.382462 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:26:50.408655 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a762bbe-f18c-494c-8fb3-7763387299f1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99889b1dab0b471d9c77887cffefc652', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:26:50.409187 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a762bbe-f18c-494c-8fb3-7763387299f1 service nova] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:26:50.409462 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a762bbe-f18c-494c-8fb3-7763387299f1 service nova] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:26:50.409727 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a762bbe-f18c-494c-8fb3-7763387299f1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:26:50.411364 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-7a762bbe-f18c-494c-8fb3-7763387299f1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:26:50.500608 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a762bbe-f18c-494c-8fb3-7763387299f1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:26:50.502849 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a762bbe-f18c-494c-8fb3-7763387299f1 service nova] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:26:50.521428 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 563/1111] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:26:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:26:51.661283 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a92b1b7-7ad0-4e92-a471-02f3958c1cc0 None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:26:51.661890 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a92b1b7-7ad0-4e92-a471-02f3958c1cc0 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:26:51.661890 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a92b1b7-7ad0-4e92-a471-02f3958c1cc0 None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:26:51.663229 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 549/1112] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 31 12:26:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 31 12:26:51.668184 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07349e4d-33b0-4a1a-a483-44e8552d0628 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:26:51.668629 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07349e4d-33b0-4a1a-a483-44e8552d0628 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:26:51.668766 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07349e4d-33b0-4a1a-a483-44e8552d0628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:26:51.745402 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-07349e4d-33b0-4a1a-a483-44e8552d0628 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:26:51.746590 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-07349e4d-33b0-4a1a-a483-44e8552d0628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:26:51.787807 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:26:51.842545 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 564/1113] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 31 12:26:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:26:51.994284 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a92b1b7-7ad0-4e92-a471-02f3958c1cc0 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:26:51.995726 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-0a92b1b7-7ad0-4e92-a471-02f3958c1cc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:26:52.139385 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a92b1b7-7ad0-4e92-a471-02f3958c1cc0 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:26:52.194226 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:26:52.254060 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ef19ee-49b6-41dd-a81c-a5f255cfd938 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99889b1dab0b471d9c77887cffefc652', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:26:52.255255 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ef19ee-49b6-41dd-a81c-a5f255cfd938 service nova] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:26:52.255536 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ef19ee-49b6-41dd-a81c-a5f255cfd938 service nova] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:26:52.255804 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ef19ee-49b6-41dd-a81c-a5f255cfd938 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:26:52.257384 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-69ef19ee-49b6-41dd-a81c-a5f255cfd938 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:26:52.348456 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ef19ee-49b6-41dd-a81c-a5f255cfd938 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:26:52.350032 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ef19ee-49b6-41dd-a81c-a5f255cfd938 service nova] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:26:52.367917 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 550/1114] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:26:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:26:53.399885 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bf8ce0-2323-4dd8-874d-c5d8c06e41ed None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:26:53.407621 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bf8ce0-2323-4dd8-874d-c5d8c06e41ed None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:26:53.407621 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bf8ce0-2323-4dd8-874d-c5d8c06e41ed None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:26:53.407621 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 565/1115] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 31 12:26:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:26:53.411437 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fb8e8d-48cd-4b88-9c1a-f20c7226b3a8 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:26:53.411883 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fb8e8d-48cd-4b88-9c1a-f20c7226b3a8 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:26:53.412037 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fb8e8d-48cd-4b88-9c1a-f20c7226b3a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:26:53.486011 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-a2fb8e8d-48cd-4b88-9c1a-f20c7226b3a8 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:26:53.487204 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-a2fb8e8d-48cd-4b88-9c1a-f20c7226b3a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:26:53.489169 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:26:53.582412 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 551/1116] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 31 12:26:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:26:53.732509 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bf8ce0-2323-4dd8-874d-c5d8c06e41ed None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:26:53.733977 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-c2bf8ce0-2323-4dd8-874d-c5d8c06e41ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:26:53.879295 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bf8ce0-2323-4dd8-874d-c5d8c06e41ed None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:26:53.907125 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:26:53.991692 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5988fad-7df1-4fbb-927b-3990d458e24c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99889b1dab0b471d9c77887cffefc652', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:26:53.992223 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5988fad-7df1-4fbb-927b-3990d458e24c service nova] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:26:53.992501 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5988fad-7df1-4fbb-927b-3990d458e24c service nova] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:26:53.992772 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5988fad-7df1-4fbb-927b-3990d458e24c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:26:53.994343 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-e5988fad-7df1-4fbb-927b-3990d458e24c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:26:54.084543 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5988fad-7df1-4fbb-927b-3990d458e24c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:26:54.086104 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5988fad-7df1-4fbb-927b-3990d458e24c service nova] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:26:54.103476 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 566/1117] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:26:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:26:56.166123 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b144e29-3ae9-45f7-b279-330033ef8239 None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:26:56.166677 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b144e29-3ae9-45f7-b279-330033ef8239 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:26:56.166677 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b144e29-3ae9-45f7-b279-330033ef8239 None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:26:56.168034 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 552/1118] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 31 12:26:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 31 12:26:56.173971 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af50a8b6-4cb1-44c0-a92d-011ff0ca7c0d None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:26:56.174390 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af50a8b6-4cb1-44c0-a92d-011ff0ca7c0d None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:26:56.174625 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af50a8b6-4cb1-44c0-a92d-011ff0ca7c0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:26:56.214754 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:26:56.240839 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-af50a8b6-4cb1-44c0-a92d-011ff0ca7c0d None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:26:56.242041 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-af50a8b6-4cb1-44c0-a92d-011ff0ca7c0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:26:56.337445 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 567/1119] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 31 12:26:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:26:56.625846 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b144e29-3ae9-45f7-b279-330033ef8239 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:26:56.627406 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-7b144e29-3ae9-45f7-b279-330033ef8239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:26:56.735179 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b144e29-3ae9-45f7-b279-330033ef8239 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:26:56.767370 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:26:56.809224 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e149deae-640c-46a1-9e47-0cfeed212c0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:26:56.809827 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e149deae-640c-46a1-9e47-0cfeed212c0c service neutron] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:26:56.810106 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e149deae-640c-46a1-9e47-0cfeed212c0c service neutron] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:26:56.810377 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e149deae-640c-46a1-9e47-0cfeed212c0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:26:56.812252 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-e149deae-640c-46a1-9e47-0cfeed212c0c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:26:56.920667 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e149deae-640c-46a1-9e47-0cfeed212c0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:26:56.922703 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e149deae-640c-46a1-9e47-0cfeed212c0c service neutron] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:26:56.940429 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 553/1120] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:26:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:26:59.230758 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd72c51-e624-4d42-8275-a9f887b97feb None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:26:59.231303 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd72c51-e624-4d42-8275-a9f887b97feb None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:26:59.231303 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd72c51-e624-4d42-8275-a9f887b97feb None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:26:59.232724 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 568/1121] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 31 12:26:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:26:59.239204 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95906fbd-9353-44f1-a348-ddca05bc9e7d None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:26:59.239640 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95906fbd-9353-44f1-a348-ddca05bc9e7d None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:26:59.239730 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95906fbd-9353-44f1-a348-ddca05bc9e7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:26:59.310809 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-95906fbd-9353-44f1-a348-ddca05bc9e7d None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:26:59.312825 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-95906fbd-9353-44f1-a348-ddca05bc9e7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:26:59.326573 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:26:59.416682 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 554/1122] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 31 12:26:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:26:59.699051 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd72c51-e624-4d42-8275-a9f887b97feb None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:26:59.700647 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-7bd72c51-e624-4d42-8275-a9f887b97feb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:26:59.803104 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd72c51-e624-4d42-8275-a9f887b97feb None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:26:59.815725 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:26:59.876898 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4881d2-0b04-4feb-a30a-3123a77ed570 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:26:59.877502 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4881d2-0b04-4feb-a30a-3123a77ed570 service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:26:59.877777 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4881d2-0b04-4feb-a30a-3123a77ed570 service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:26:59.878048 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4881d2-0b04-4feb-a30a-3123a77ed570 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:26:59.879648 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-dc4881d2-0b04-4feb-a30a-3123a77ed570 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:26:59.988169 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4881d2-0b04-4feb-a30a-3123a77ed570 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:26:59.989504 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4881d2-0b04-4feb-a30a-3123a77ed570 service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:27:00.006838 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 569/1123] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:26:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:27:02.476709 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9c7e1c-2163-4e4a-af14-b4628085a0de None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:27:02.477249 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9c7e1c-2163-4e4a-af14-b4628085a0de None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:27:02.477249 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9c7e1c-2163-4e4a-af14-b4628085a0de None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:27:02.478497 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 555/1124] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 31 12:27:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:27:02.484257 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30f7749-a98a-4eb1-bd96-cf83cca04421 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:27:02.484768 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30f7749-a98a-4eb1-bd96-cf83cca04421 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:27:02.484875 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30f7749-a98a-4eb1-bd96-cf83cca04421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:27:02.532743 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:27:02.550467 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-a30f7749-a98a-4eb1-bd96-cf83cca04421 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:27:02.551692 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-a30f7749-a98a-4eb1-bd96-cf83cca04421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:27:02.646262 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 570/1125] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 31 12:27:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:27:02.938869 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9c7e1c-2163-4e4a-af14-b4628085a0de None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:27:02.940448 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-0e9c7e1c-2163-4e4a-af14-b4628085a0de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:27:03.045477 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9c7e1c-2163-4e4a-af14-b4628085a0de None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:27:03.085279 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:27:03.119621 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33aa1a9-1730-40b6-9785-9b8c3f2c3a6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:27:03.120217 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33aa1a9-1730-40b6-9785-9b8c3f2c3a6b service neutron] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:27:03.120491 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33aa1a9-1730-40b6-9785-9b8c3f2c3a6b service neutron] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:27:03.120754 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33aa1a9-1730-40b6-9785-9b8c3f2c3a6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:27:03.122346 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-c33aa1a9-1730-40b6-9785-9b8c3f2c3a6b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:27:03.230849 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33aa1a9-1730-40b6-9785-9b8c3f2c3a6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:27:03.232228 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33aa1a9-1730-40b6-9785-9b8c3f2c3a6b service neutron] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:27:03.250446 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 556/1126] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:27:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:27:05.500963 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59f50eb-c473-4b63-957f-913ec3ab018f None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:27:05.501483 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59f50eb-c473-4b63-957f-913ec3ab018f None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:27:05.501483 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59f50eb-c473-4b63-957f-913ec3ab018f None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:27:05.502780 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 571/1127] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 31 12:27:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 31 12:27:05.508088 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212afe30-0bca-4b7a-90d9-e65d0b779840 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:27:05.508479 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212afe30-0bca-4b7a-90d9-e65d0b779840 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:27:05.508632 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212afe30-0bca-4b7a-90d9-e65d0b779840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:27:05.573854 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-212afe30-0bca-4b7a-90d9-e65d0b779840 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:27:05.575047 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-212afe30-0bca-4b7a-90d9-e65d0b779840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:27:05.596984 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:27:05.672367 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 557/1128] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 31 12:27:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:27:05.955733 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59f50eb-c473-4b63-957f-913ec3ab018f None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:27:05.957251 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-b59f50eb-c473-4b63-957f-913ec3ab018f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:27:06.060036 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59f50eb-c473-4b63-957f-913ec3ab018f None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:27:06.080406 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:27:06.132983 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac707aad-2af3-4695-b822-ea3c92f49378 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:27:06.133589 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac707aad-2af3-4695-b822-ea3c92f49378 service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:27:06.133867 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac707aad-2af3-4695-b822-ea3c92f49378 service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:27:06.134137 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac707aad-2af3-4695-b822-ea3c92f49378 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:27:06.135743 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-ac707aad-2af3-4695-b822-ea3c92f49378 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:27:06.243655 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac707aad-2af3-4695-b822-ea3c92f49378 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:27:06.245003 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac707aad-2af3-4695-b822-ea3c92f49378 service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:27:06.262939 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 572/1129] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:27:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:27:08.474523 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0fe771-77c3-4829-8c2c-8fff7b3d59a5 None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:27:08.475060 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0fe771-77c3-4829-8c2c-8fff7b3d59a5 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:27:08.475060 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0fe771-77c3-4829-8c2c-8fff7b3d59a5 None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:27:08.476387 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 558/1130] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 31 12:27:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:27:08.481723 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3c96a3-6e90-4626-bc4e-5e732b1e104e None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:27:08.482163 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3c96a3-6e90-4626-bc4e-5e732b1e104e None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:27:08.482269 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3c96a3-6e90-4626-bc4e-5e732b1e104e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:27:08.544899 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:27:08.548220 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-5e3c96a3-6e90-4626-bc4e-5e732b1e104e None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:27:08.549958 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-5e3c96a3-6e90-4626-bc4e-5e732b1e104e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:27:08.644426 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 573/1131] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 31 12:27:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:27:08.930735 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0fe771-77c3-4829-8c2c-8fff7b3d59a5 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:27:08.932300 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-6f0fe771-77c3-4829-8c2c-8fff7b3d59a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:27:09.036897 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0fe771-77c3-4829-8c2c-8fff7b3d59a5 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:27:09.085979 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:27:09.110348 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6eb17b-8f33-42ec-b673-e83b45a4e7a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:27:09.110948 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6eb17b-8f33-42ec-b673-e83b45a4e7a5 service neutron] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:27:09.111253 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6eb17b-8f33-42ec-b673-e83b45a4e7a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:27:09.111523 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6eb17b-8f33-42ec-b673-e83b45a4e7a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:27:09.113101 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-ec6eb17b-8f33-42ec-b673-e83b45a4e7a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:27:09.221949 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6eb17b-8f33-42ec-b673-e83b45a4e7a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:27:09.223316 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6eb17b-8f33-42ec-b673-e83b45a4e7a5 service neutron] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:27:09.241267 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 559/1132] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:27:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:27:11.452668 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ed6d90-bd2d-460c-bb3c-0da282d58e07 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:27:11.453179 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ed6d90-bd2d-460c-bb3c-0da282d58e07 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:27:11.453179 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ed6d90-bd2d-460c-bb3c-0da282d58e07 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:27:11.454451 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 574/1133] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 31 12:27:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:27:11.459927 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99934e2b-9a02-4108-98c5-6ca470a59f8b None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:27:11.460356 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99934e2b-9a02-4108-98c5-6ca470a59f8b None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:27:11.460476 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99934e2b-9a02-4108-98c5-6ca470a59f8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:27:11.525393 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-99934e2b-9a02-4108-98c5-6ca470a59f8b None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:27:11.526603 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-99934e2b-9a02-4108-98c5-6ca470a59f8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:27:11.555607 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:27:11.622902 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 560/1134] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 31 12:27:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:27:11.913537 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ed6d90-bd2d-460c-bb3c-0da282d58e07 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:27:11.915089 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-85ed6d90-bd2d-460c-bb3c-0da282d58e07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:27:12.017842 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ed6d90-bd2d-460c-bb3c-0da282d58e07 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:27:12.048151 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:27:12.093082 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33baa4ea-39f9-468c-9cfa-ca15f891707d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:27:12.093683 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33baa4ea-39f9-468c-9cfa-ca15f891707d service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:27:12.093951 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33baa4ea-39f9-468c-9cfa-ca15f891707d service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:27:12.094221 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33baa4ea-39f9-468c-9cfa-ca15f891707d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:27:12.095801 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-33baa4ea-39f9-468c-9cfa-ca15f891707d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:27:12.204638 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33baa4ea-39f9-468c-9cfa-ca15f891707d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:27:12.206120 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33baa4ea-39f9-468c-9cfa-ca15f891707d service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:27:12.224433 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 575/1135] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:27:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:27:14.430549 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b26ac5-a4be-4628-b970-23c70e425c12 None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:27:14.431087 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b26ac5-a4be-4628-b970-23c70e425c12 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:27:14.431087 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b26ac5-a4be-4628-b970-23c70e425c12 None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:27:14.432470 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 561/1136] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 31 12:27:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:27:14.437908 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b91d7a-2959-48c0-8bc1-04c9d24e2a15 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:27:14.438322 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b91d7a-2959-48c0-8bc1-04c9d24e2a15 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:27:14.438501 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b91d7a-2959-48c0-8bc1-04c9d24e2a15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:27:14.503118 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-e1b91d7a-2959-48c0-8bc1-04c9d24e2a15 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:27:14.504327 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-e1b91d7a-2959-48c0-8bc1-04c9d24e2a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:27:14.512389 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:27:14.600208 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 576/1137] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 31 12:27:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:27:14.881218 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b26ac5-a4be-4628-b970-23c70e425c12 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:27:14.882755 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-14b26ac5-a4be-4628-b970-23c70e425c12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:27:14.986791 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b26ac5-a4be-4628-b970-23c70e425c12 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:27:15.042499 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:27:15.061270 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93933955-2605-46f4-8a47-c1804f29575e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:27:15.061869 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93933955-2605-46f4-8a47-c1804f29575e service neutron] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:27:15.062147 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93933955-2605-46f4-8a47-c1804f29575e service neutron] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:27:15.062417 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93933955-2605-46f4-8a47-c1804f29575e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:27:15.064022 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-93933955-2605-46f4-8a47-c1804f29575e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:27:15.171623 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93933955-2605-46f4-8a47-c1804f29575e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:27:15.173040 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93933955-2605-46f4-8a47-c1804f29575e service neutron] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:27:15.190432 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 562/1138] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:27:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:27:17.377437 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b55948-8982-49bf-8bf5-0087c17d43e2 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:27:17.378024 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b55948-8982-49bf-8bf5-0087c17d43e2 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:27:17.378024 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b55948-8982-49bf-8bf5-0087c17d43e2 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:27:17.379350 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 577/1139] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 31 12:27:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:27:17.384572 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc208c8e-ee63-4fee-8223-0ed46533d365 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:27:17.385029 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc208c8e-ee63-4fee-8223-0ed46533d365 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:27:17.385164 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc208c8e-ee63-4fee-8223-0ed46533d365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:27:17.452422 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-dc208c8e-ee63-4fee-8223-0ed46533d365 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:27:17.453612 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-dc208c8e-ee63-4fee-8223-0ed46533d365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:27:17.491332 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:27:17.548460 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 563/1140] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 31 12:27:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:27:17.835028 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b55948-8982-49bf-8bf5-0087c17d43e2 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:27:17.836577 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-78b55948-8982-49bf-8bf5-0087c17d43e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:27:17.941776 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b55948-8982-49bf-8bf5-0087c17d43e2 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:27:17.979521 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:27:18.015659 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fafde1-5306-42bc-997d-a1d2e934e1ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:27:18.016260 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fafde1-5306-42bc-997d-a1d2e934e1ee service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:27:18.016539 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fafde1-5306-42bc-997d-a1d2e934e1ee service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:27:18.016807 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fafde1-5306-42bc-997d-a1d2e934e1ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:27:18.018376 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-44fafde1-5306-42bc-997d-a1d2e934e1ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:27:18.126817 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fafde1-5306-42bc-997d-a1d2e934e1ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:27:18.128189 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fafde1-5306-42bc-997d-a1d2e934e1ee service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:27:18.146419 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 578/1141] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:27:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:27:20.349761 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b46f838-ed89-4568-a039-436f6d0bdf65 None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:27:20.350321 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b46f838-ed89-4568-a039-436f6d0bdf65 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:27:20.350321 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b46f838-ed89-4568-a039-436f6d0bdf65 None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:27:20.351596 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 564/1142] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 31 12:27:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:27:20.356791 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08159349-7912-4a24-86db-5e2240a03c24 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:27:20.357220 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08159349-7912-4a24-86db-5e2240a03c24 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:27:20.357386 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08159349-7912-4a24-86db-5e2240a03c24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:27:20.422804 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-08159349-7912-4a24-86db-5e2240a03c24 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:27:20.424029 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-08159349-7912-4a24-86db-5e2240a03c24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:27:20.441532 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:27:20.520550 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 579/1143] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 31 12:27:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:27:20.802564 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b46f838-ed89-4568-a039-436f6d0bdf65 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:27:20.804107 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-2b46f838-ed89-4568-a039-436f6d0bdf65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:27:20.907714 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b46f838-ed89-4568-a039-436f6d0bdf65 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:27:20.974114 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:27:20.981122 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d693fc6-c881-4607-a1e6-bba8f3d2f131 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:27:20.981719 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d693fc6-c881-4607-a1e6-bba8f3d2f131 service neutron] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:27:20.981991 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d693fc6-c881-4607-a1e6-bba8f3d2f131 service neutron] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:27:20.983513 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d693fc6-c881-4607-a1e6-bba8f3d2f131 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:27:20.985081 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-2d693fc6-c881-4607-a1e6-bba8f3d2f131 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:27:21.094745 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d693fc6-c881-4607-a1e6-bba8f3d2f131 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:27:21.096116 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d693fc6-c881-4607-a1e6-bba8f3d2f131 service neutron] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:27:21.114091 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 565/1144] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:27:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:27:23.254506 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338ccf8f-4540-43f5-af7b-6fadb25a84c1 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:27:23.255065 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338ccf8f-4540-43f5-af7b-6fadb25a84c1 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:27:23.255065 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338ccf8f-4540-43f5-af7b-6fadb25a84c1 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:27:23.256434 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 580/1145] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 31 12:27:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:27:23.261588 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac82891a-75d4-4149-bf7b-2d4bfcb479ef None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:27:23.261999 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac82891a-75d4-4149-bf7b-2d4bfcb479ef None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:27:23.262126 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac82891a-75d4-4149-bf7b-2d4bfcb479ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:27:23.329929 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-ac82891a-75d4-4149-bf7b-2d4bfcb479ef None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:27:23.331036 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-ac82891a-75d4-4149-bf7b-2d4bfcb479ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:27:23.378270 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:27:23.426583 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 566/1146] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 31 12:27:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:27:23.709994 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338ccf8f-4540-43f5-af7b-6fadb25a84c1 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:27:23.711552 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-338ccf8f-4540-43f5-af7b-6fadb25a84c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:27:23.814323 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338ccf8f-4540-43f5-af7b-6fadb25a84c1 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:27:23.862098 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:27:23.887173 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567fcf29-975b-46e1-8e4b-0f486402603e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:27:23.887795 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567fcf29-975b-46e1-8e4b-0f486402603e service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:27:23.888079 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567fcf29-975b-46e1-8e4b-0f486402603e service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:27:23.888375 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567fcf29-975b-46e1-8e4b-0f486402603e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:27:23.889942 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-567fcf29-975b-46e1-8e4b-0f486402603e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:27:24.002796 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567fcf29-975b-46e1-8e4b-0f486402603e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:27:24.004148 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567fcf29-975b-46e1-8e4b-0f486402603e service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:27:24.022091 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 581/1147] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:27:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:27:33.425051 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=22 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:27:34.019617 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=66 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:27:44.010346 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09fc2bf-c0ec-4be0-8628-b345ece1afe2 None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:27:44.010913 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09fc2bf-c0ec-4be0-8628-b345ece1afe2 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:27:44.010913 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09fc2bf-c0ec-4be0-8628-b345ece1afe2 None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:27:44.012340 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 567/1148] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:27:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:27:44.018787 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a810f0c-de3d-4bcd-b78e-f34ddf3c0590 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:27:44.019209 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a810f0c-de3d-4bcd-b78e-f34ddf3c0590 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:27:44.019385 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a810f0c-de3d-4bcd-b78e-f34ddf3c0590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:27:44.088022 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-6a810f0c-de3d-4bcd-b78e-f34ddf3c0590 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:27:44.089210 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-6a810f0c-de3d-4bcd-b78e-f34ddf3c0590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:27:44.182436 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 582/1149] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:27:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:27:44.484083 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403cdab1-352a-4226-a157-1d924c08c544 None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:27:44.484361 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403cdab1-352a-4226-a157-1d924c08c544 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:27:44.484628 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403cdab1-352a-4226-a157-1d924c08c544 None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:27:44.485981 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 568/1150] 10.222.0.15 () {54 vars in 962 bytes} [Mon Oct 31 12:27:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:27:44.490903 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a22b40b-8380-402a-8cac-e4ee571f6d4e None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:27:44.491388 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a22b40b-8380-402a-8cac-e4ee571f6d4e None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:27:44.491663 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a22b40b-8380-402a-8cac-e4ee571f6d4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:27:44.565728 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-2a22b40b-8380-402a-8cac-e4ee571f6d4e None None] MFA Rules not processed for user `4feaa01c5ed64b65b10d4dece823702c`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:27:44.566952 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-2a22b40b-8380-402a-8cac-e4ee571f6d4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:27:44.586662 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:27:44.641310 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 583/1151] 10.222.0.15 () {58 vars in 1059 bytes} [Mon Oct 31 12:27:44 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:27:44.648942 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403cdab1-352a-4226-a157-1d924c08c544 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:27:44.650436 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-403cdab1-352a-4226-a157-1d924c08c544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:27:44.753602 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403cdab1-352a-4226-a157-1d924c08c544 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:27:44.825818 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1b970d-0f0e-4012-88f7-99dc11ab1d1d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4feaa01c5ed64b65b10d4dece823702c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:27:44.826416 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1b970d-0f0e-4012-88f7-99dc11ab1d1d service glance] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:27:44.826692 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1b970d-0f0e-4012-88f7-99dc11ab1d1d service glance] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:27:44.827553 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1b970d-0f0e-4012-88f7-99dc11ab1d1d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:27:44.829120 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-3f1b970d-0f0e-4012-88f7-99dc11ab1d1d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:27:44.881906 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:27:44.936024 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1b970d-0f0e-4012-88f7-99dc11ab1d1d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:27:44.938055 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1b970d-0f0e-4012-88f7-99dc11ab1d1d service glance] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:27:44.956705 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 569/1152] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:27:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:27:47.185028 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d828f12-afad-4484-84b5-f4af5b75447d None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:27:47.185588 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d828f12-afad-4484-84b5-f4af5b75447d None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:27:47.185588 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d828f12-afad-4484-84b5-f4af5b75447d None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:27:47.187505 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 584/1153] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:27:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:27:47.192631 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7ac058-9bc1-40ce-a898-d65407b2c9b0 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:27:47.193020 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7ac058-9bc1-40ce-a898-d65407b2c9b0 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:27:47.193170 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7ac058-9bc1-40ce-a898-d65407b2c9b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:27:47.258140 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-8a7ac058-9bc1-40ce-a898-d65407b2c9b0 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:27:47.259353 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-8a7ac058-9bc1-40ce-a898-d65407b2c9b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:27:47.357729 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 570/1154] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:27:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:27:47.629516 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d828f12-afad-4484-84b5-f4af5b75447d None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:27:47.631009 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-3d828f12-afad-4484-84b5-f4af5b75447d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:27:47.758359 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d828f12-afad-4484-84b5-f4af5b75447d None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:27:47.809123 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:27:47.853452 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112f9f3b-a7b4-4bd6-9726-f212051f7de7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:27:47.854061 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112f9f3b-a7b4-4bd6-9726-f212051f7de7 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:27:47.854341 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112f9f3b-a7b4-4bd6-9726-f212051f7de7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:27:47.854610 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112f9f3b-a7b4-4bd6-9726-f212051f7de7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:27:47.855275 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112f9f3b-a7b4-4bd6-9726-f212051f7de7 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:27:47.856394 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: WARNING py.warnings [None req-112f9f3b-a7b4-4bd6-9726-f212051f7de7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 31 12:27:47.856394 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: warnings.warn(msg) Oct 31 12:27:47.856394 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: Oct 31 12:27:47.857518 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112f9f3b-a7b4-4bd6-9726-f212051f7de7 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:27:47.876774 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 585/1155] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Oct 31 12:27:47 2022] GET /identity/v3/projects => generated 2553 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:27:50.050114 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3216db7-9c44-42de-b4d2-c8d3f3712887 None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:27:50.050573 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3216db7-9c44-42de-b4d2-c8d3f3712887 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:27:50.050676 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3216db7-9c44-42de-b4d2-c8d3f3712887 None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:27:50.052079 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 571/1156] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:27:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:27:50.057216 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35928e7-4cc2-4c2b-b3ac-97724174586d None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:27:50.057645 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35928e7-4cc2-4c2b-b3ac-97724174586d None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:27:50.057766 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35928e7-4cc2-4c2b-b3ac-97724174586d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:27:50.125278 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-f35928e7-4cc2-4c2b-b3ac-97724174586d None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:27:50.126480 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-f35928e7-4cc2-4c2b-b3ac-97724174586d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:27:50.223459 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 586/1157] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:27:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:27:50.521625 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3216db7-9c44-42de-b4d2-c8d3f3712887 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:27:50.523133 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-b3216db7-9c44-42de-b4d2-c8d3f3712887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:27:50.533432 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:27:50.666778 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3216db7-9c44-42de-b4d2-c8d3f3712887 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:27:50.745434 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:27:50.778051 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8fd28a-6d26-470b-ae6e-35cef35f87b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99889b1dab0b471d9c77887cffefc652', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:27:50.778634 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8fd28a-6d26-470b-ae6e-35cef35f87b8 service nova] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:27:50.778909 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8fd28a-6d26-470b-ae6e-35cef35f87b8 service nova] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:27:50.779183 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8fd28a-6d26-470b-ae6e-35cef35f87b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:27:50.780797 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-db8fd28a-6d26-470b-ae6e-35cef35f87b8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:27:50.867267 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8fd28a-6d26-470b-ae6e-35cef35f87b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:27:50.868821 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8fd28a-6d26-470b-ae6e-35cef35f87b8 service nova] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:27:50.886608 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 572/1158] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:27:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:27:53.327481 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d87a2f-4491-42f9-ad46-1d623373ed51 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:27:53.328010 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d87a2f-4491-42f9-ad46-1d623373ed51 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:27:53.328010 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d87a2f-4491-42f9-ad46-1d623373ed51 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:27:53.329397 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 587/1159] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:27:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:27:53.334807 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b868ba1-1782-42da-98b7-aa200fff5fbd None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:27:53.335263 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b868ba1-1782-42da-98b7-aa200fff5fbd None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:27:53.335371 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b868ba1-1782-42da-98b7-aa200fff5fbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:27:53.401909 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-4b868ba1-1782-42da-98b7-aa200fff5fbd None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:27:53.403103 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-4b868ba1-1782-42da-98b7-aa200fff5fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:27:53.448357 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:27:53.500018 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 573/1160] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:27:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:27:53.788480 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d87a2f-4491-42f9-ad46-1d623373ed51 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:27:53.790035 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-96d87a2f-4491-42f9-ad46-1d623373ed51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:27:53.814311 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:27:53.891723 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d87a2f-4491-42f9-ad46-1d623373ed51 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:27:53.965297 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eac43dd-09f7-42c4-a785-416fc714679f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:27:53.965922 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eac43dd-09f7-42c4-a785-416fc714679f service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:27:53.966192 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eac43dd-09f7-42c4-a785-416fc714679f service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:27:53.966454 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eac43dd-09f7-42c4-a785-416fc714679f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:27:53.968621 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-9eac43dd-09f7-42c4-a785-416fc714679f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:27:54.048357 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:27:54.082903 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eac43dd-09f7-42c4-a785-416fc714679f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:27:54.084354 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eac43dd-09f7-42c4-a785-416fc714679f service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:27:54.102214 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 588/1161] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:27:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:27:56.323027 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da9055b-a9ad-4892-b3dc-e2930ff5d318 None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:27:56.323618 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da9055b-a9ad-4892-b3dc-e2930ff5d318 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:27:56.323618 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da9055b-a9ad-4892-b3dc-e2930ff5d318 None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:27:56.324881 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 574/1162] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:27:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:27:56.330258 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dacd673-57f4-43a9-ac04-3f23584c3d9f None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:27:56.330704 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dacd673-57f4-43a9-ac04-3f23584c3d9f None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:27:56.330811 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dacd673-57f4-43a9-ac04-3f23584c3d9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:27:56.396035 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-0dacd673-57f4-43a9-ac04-3f23584c3d9f None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:27:56.397287 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-0dacd673-57f4-43a9-ac04-3f23584c3d9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:27:56.492003 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 589/1163] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:27:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:27:56.780851 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da9055b-a9ad-4892-b3dc-e2930ff5d318 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:27:56.783204 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-1da9055b-a9ad-4892-b3dc-e2930ff5d318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:27:56.882829 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da9055b-a9ad-4892-b3dc-e2930ff5d318 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:27:56.953913 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:27:56.955571 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea710b04-be0b-42e1-b68d-7dfbce0ed7b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:27:56.956223 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea710b04-be0b-42e1-b68d-7dfbce0ed7b9 service neutron] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:27:56.956687 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea710b04-be0b-42e1-b68d-7dfbce0ed7b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:27:56.957168 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea710b04-be0b-42e1-b68d-7dfbce0ed7b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:27:56.959021 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-ea710b04-be0b-42e1-b68d-7dfbce0ed7b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:27:57.068458 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea710b04-be0b-42e1-b68d-7dfbce0ed7b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:27:57.069859 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea710b04-be0b-42e1-b68d-7dfbce0ed7b9 service neutron] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:27:57.087842 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 575/1164] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:27:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:27:59.385756 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408a9d91-9c9c-493a-8a4c-916e64165577 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:27:59.386282 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408a9d91-9c9c-493a-8a4c-916e64165577 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:27:59.386282 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408a9d91-9c9c-493a-8a4c-916e64165577 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:27:59.387588 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 590/1165] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:27:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:27:59.393063 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0737221e-1ac3-49a6-9d63-c89066b7f118 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:27:59.393460 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0737221e-1ac3-49a6-9d63-c89066b7f118 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:27:59.393612 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0737221e-1ac3-49a6-9d63-c89066b7f118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:27:59.458614 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-0737221e-1ac3-49a6-9d63-c89066b7f118 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:27:59.459836 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-0737221e-1ac3-49a6-9d63-c89066b7f118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:27:59.514468 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:27:59.563623 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 576/1166] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:27:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:27:59.858530 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408a9d91-9c9c-493a-8a4c-916e64165577 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:27:59.860917 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-408a9d91-9c9c-493a-8a4c-916e64165577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:27:59.896495 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:27:59.964361 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408a9d91-9c9c-493a-8a4c-916e64165577 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:28:00.039681 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758ab441-be6e-4279-a911-e8b6717650b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:28:00.040295 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758ab441-be6e-4279-a911-e8b6717650b2 service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:28:00.040570 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758ab441-be6e-4279-a911-e8b6717650b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:28:00.040842 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758ab441-be6e-4279-a911-e8b6717650b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:28:00.042429 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-758ab441-be6e-4279-a911-e8b6717650b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:28:00.122577 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:28:00.151587 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758ab441-be6e-4279-a911-e8b6717650b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:28:00.152997 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758ab441-be6e-4279-a911-e8b6717650b2 service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:28:00.171351 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 591/1167] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:27:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:28:02.572284 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb8c8cd-f676-44e1-8173-02c93fa96739 None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:28:02.572821 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb8c8cd-f676-44e1-8173-02c93fa96739 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:28:02.572821 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb8c8cd-f676-44e1-8173-02c93fa96739 None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:28:02.574173 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 577/1168] 10.222.0.15 () {54 vars in 927 bytes} [Mon Oct 31 12:28:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:28:02.579899 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bec8d86-67cd-4e05-8248-d6fd236ee84b None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:28:02.580338 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bec8d86-67cd-4e05-8248-d6fd236ee84b None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:28:02.580434 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bec8d86-67cd-4e05-8248-d6fd236ee84b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:28:02.646511 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-7bec8d86-67cd-4e05-8248-d6fd236ee84b None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:28:02.647725 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-7bec8d86-67cd-4e05-8248-d6fd236ee84b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:28:02.743212 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 592/1169] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Oct 31 12:28:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:28:03.043257 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb8c8cd-f676-44e1-8173-02c93fa96739 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:28:03.044803 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-0cb8c8cd-f676-44e1-8173-02c93fa96739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:28:03.147504 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb8c8cd-f676-44e1-8173-02c93fa96739 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:28:03.220680 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef773291-fe38-4251-aa9f-dd8175faa46c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:28:03.221278 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef773291-fe38-4251-aa9f-dd8175faa46c service neutron] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:28:03.221551 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef773291-fe38-4251-aa9f-dd8175faa46c service neutron] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:28:03.221816 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef773291-fe38-4251-aa9f-dd8175faa46c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:28:03.223406 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-ef773291-fe38-4251-aa9f-dd8175faa46c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:28:03.231736 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:28:03.331524 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef773291-fe38-4251-aa9f-dd8175faa46c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:28:03.332866 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef773291-fe38-4251-aa9f-dd8175faa46c service neutron] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:28:03.350333 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 578/1170] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:28:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:28:12.741662 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=82 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:28:13.348175 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=50 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:29:16.151034 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7370a13-a0d7-454e-a89f-6325f7afa2af None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:29:16.151663 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7370a13-a0d7-454e-a89f-6325f7afa2af None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:29:16.151663 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7370a13-a0d7-454e-a89f-6325f7afa2af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:29:16.225499 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-a7370a13-a0d7-454e-a89f-6325f7afa2af None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:29:16.226696 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-a7370a13-a0d7-454e-a89f-6325f7afa2af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:29:16.321120 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 593/1171] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:29:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:29:16.329541 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140b5a96-92d3-4555-9dce-00eaa3ea493f None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:29:16.330038 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140b5a96-92d3-4555-9dce-00eaa3ea493f None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:29:16.330134 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140b5a96-92d3-4555-9dce-00eaa3ea493f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:29:16.406670 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-140b5a96-92d3-4555-9dce-00eaa3ea493f None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:29:16.407811 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-140b5a96-92d3-4555-9dce-00eaa3ea493f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:29:16.503767 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 579/1172] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:29:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:29:16.508357 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7370a13-a0d7-454e-a89f-6325f7afa2af None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:29:16.509586 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-a7370a13-a0d7-454e-a89f-6325f7afa2af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:29:16.521894 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7370a13-a0d7-454e-a89f-6325f7afa2af None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:29:16.581886 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:29:16.615626 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d405457d-0cc9-44d5-bfcc-02a06491f2cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:29:16.616219 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d405457d-0cc9-44d5-bfcc-02a06491f2cd admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:29:16.616493 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d405457d-0cc9-44d5-bfcc-02a06491f2cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:29:16.616759 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d405457d-0cc9-44d5-bfcc-02a06491f2cd admin admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:29:16.617391 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d405457d-0cc9-44d5-bfcc-02a06491f2cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:29:16.617687 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d405457d-0cc9-44d5-bfcc-02a06491f2cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:29:16.618680 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: WARNING py.warnings [None req-d405457d-0cc9-44d5-bfcc-02a06491f2cd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 31 12:29:16.618680 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: warnings.warn(msg) Oct 31 12:29:16.618680 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: Oct 31 12:29:16.619593 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d405457d-0cc9-44d5-bfcc-02a06491f2cd admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:29:16.631243 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 594/1173] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Oct 31 12:29:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:29:16.640536 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140b5a96-92d3-4555-9dce-00eaa3ea493f None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:29:16.649674 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140b5a96-92d3-4555-9dce-00eaa3ea493f None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:29:16.714374 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:29:16.754117 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7635b0-48a1-4397-a2c2-f28c47aeb853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:29:16.754727 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7635b0-48a1-4397-a2c2-f28c47aeb853 admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:29:16.755003 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7635b0-48a1-4397-a2c2-f28c47aeb853 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:29:16.755303 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7635b0-48a1-4397-a2c2-f28c47aeb853 admin admin] PATH_INFO: `/v3/projects` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:29:16.755926 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7635b0-48a1-4397-a2c2-f28c47aeb853 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:29:16.757040 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: WARNING py.warnings [None req-5b7635b0-48a1-4397-a2c2-f28c47aeb853 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 31 12:29:16.757040 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: warnings.warn(msg) Oct 31 12:29:16.757040 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: Oct 31 12:29:16.758208 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7635b0-48a1-4397-a2c2-f28c47aeb853 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:29:16.814013 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 580/1174] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:29:16 2022] POST /identity/v3/projects => generated 375 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:29:16.818160 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d405457d-0cc9-44d5-bfcc-02a06491f2cd admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:29:16.819755 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-d405457d-0cc9-44d5-bfcc-02a06491f2cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:29:16.852748 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d405457d-0cc9-44d5-bfcc-02a06491f2cd admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:29:16.944232 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e21f9c-655f-417c-9043-4cebf16ce238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:29:16.944833 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e21f9c-655f-417c-9043-4cebf16ce238 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:29:16.945111 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e21f9c-655f-417c-9043-4cebf16ce238 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:29:16.945383 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e21f9c-655f-417c-9043-4cebf16ce238 admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:29:16.946004 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e21f9c-655f-417c-9043-4cebf16ce238 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:29:16.946998 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: WARNING py.warnings [None req-d3e21f9c-655f-417c-9043-4cebf16ce238 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 31 12:29:16.946998 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: warnings.warn(msg) Oct 31 12:29:16.946998 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: Oct 31 12:29:16.948221 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e21f9c-655f-417c-9043-4cebf16ce238 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:29:16.991856 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 595/1175] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:29:16 2022] POST /identity/v3/users => generated 288 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 31 12:29:16.995900 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7635b0-48a1-4397-a2c2-f28c47aeb853 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:29:16.997397 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-5b7635b0-48a1-4397-a2c2-f28c47aeb853 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:29:17.061875 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7635b0-48a1-4397-a2c2-f28c47aeb853 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:29:17.143896 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:29:17.154694 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a488e592-34e0-43c5-98c9-589595e6e7da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:29:17.155325 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a488e592-34e0-43c5-98c9-589595e6e7da admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:29:17.155597 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a488e592-34e0-43c5-98c9-589595e6e7da admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:29:17.155859 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a488e592-34e0-43c5-98c9-589595e6e7da admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:29:17.156993 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a488e592-34e0-43c5-98c9-589595e6e7da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:29:17.158585 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a488e592-34e0-43c5-98c9-589595e6e7da admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:29:17.170711 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 581/1176] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:29:16 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 31 12:29:17.175389 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e21f9c-655f-417c-9043-4cebf16ce238 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:29:17.176903 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-d3e21f9c-655f-417c-9043-4cebf16ce238 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:29:17.188443 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e21f9c-655f-417c-9043-4cebf16ce238 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:29:17.213053 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:29:17.284012 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b74e9f-ab51-4bbc-b436-38dd397ae558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:29:17.284614 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b74e9f-ab51-4bbc-b436-38dd397ae558 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:29:17.284885 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b74e9f-ab51-4bbc-b436-38dd397ae558 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:29:17.285147 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b74e9f-ab51-4bbc-b436-38dd397ae558 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:29:17.285805 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b74e9f-ab51-4bbc-b436-38dd397ae558 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:29:17.287407 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b74e9f-ab51-4bbc-b436-38dd397ae558 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:29:17.299359 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 596/1177] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:29:17 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:29:17.303685 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a488e592-34e0-43c5-98c9-589595e6e7da admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:29:17.305191 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-a488e592-34e0-43c5-98c9-589595e6e7da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:29:17.339722 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a488e592-34e0-43c5-98c9-589595e6e7da admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:29:17.433918 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ef5666-8953-4cdf-955e-05c202c6d6f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:29:17.434573 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ef5666-8953-4cdf-955e-05c202c6d6f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:29:17.434848 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ef5666-8953-4cdf-955e-05c202c6d6f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:29:17.435114 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ef5666-8953-4cdf-955e-05c202c6d6f5 admin admin] PATH_INFO: `/v3/projects/583d58e27a3d4577b832d7a0b58f3b11/users/cd86e149510b4a1eb7d267a9a6a418ba/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:29:17.459306 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ef5666-8953-4cdf-955e-05c202c6d6f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=583d58e27a3d4577b832d7a0b58f3b11, user_id=cd86e149510b4a1eb7d267a9a6a418ba, role_id=ca1a5695f8bd499aa0b8cedaeecd2cc6)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:29:17.461942 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ef5666-8953-4cdf-955e-05c202c6d6f5 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:29:17.486402 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 582/1178] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:29:17 2022] PUT /identity/v3/projects/583d58e27a3d4577b832d7a0b58f3b11/users/cd86e149510b4a1eb7d267a9a6a418ba/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:29:17.497366 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96779d8f-585f-45c6-a354-f93be8c33f02 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:29:17.497677 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96779d8f-585f-45c6-a354-f93be8c33f02 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:29:17.497976 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96779d8f-585f-45c6-a354-f93be8c33f02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:29:17.555355 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-96779d8f-585f-45c6-a354-f93be8c33f02 None None] MFA Rules not processed for user `cd86e149510b4a1eb7d267a9a6a418ba`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:29:17.556469 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-96779d8f-585f-45c6-a354-f93be8c33f02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:29:17.633076 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 597/1179] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:29:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 31 12:29:17.647247 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d07ae2-43cd-434b-8f55-60c1bd0ba242 None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:29:17.647533 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d07ae2-43cd-434b-8f55-60c1bd0ba242 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:29:17.647807 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d07ae2-43cd-434b-8f55-60c1bd0ba242 None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:29:17.649161 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 583/1180] 10.222.0.15 () {54 vars in 963 bytes} [Mon Oct 31 12:29:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:29:17.653994 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fedba2-7f36-47bf-8edc-81bd69bcfb62 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:29:17.654294 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fedba2-7f36-47bf-8edc-81bd69bcfb62 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:29:17.654565 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fedba2-7f36-47bf-8edc-81bd69bcfb62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:29:17.685615 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:29:17.731126 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-51fedba2-7f36-47bf-8edc-81bd69bcfb62 None None] MFA Rules not processed for user `20d6d7ea077d49e8a66c6c6762499702`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:29:17.732348 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-51fedba2-7f36-47bf-8edc-81bd69bcfb62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:29:17.807095 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 598/1181] 10.222.0.15 () {58 vars in 1060 bytes} [Mon Oct 31 12:29:17 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:29:17.816753 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d07ae2-43cd-434b-8f55-60c1bd0ba242 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:29:17.818249 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-59d07ae2-43cd-434b-8f55-60c1bd0ba242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:29:17.867947 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:29:17.921201 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d07ae2-43cd-434b-8f55-60c1bd0ba242 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:29:17.996921 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c699fc56-6339-4cb6-a746-73e1f528a42f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d6d7ea077d49e8a66c6c6762499702', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:29:17.997521 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c699fc56-6339-4cb6-a746-73e1f528a42f service cinder] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:29:17.997795 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c699fc56-6339-4cb6-a746-73e1f528a42f service cinder] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:29:17.998064 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c699fc56-6339-4cb6-a746-73e1f528a42f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:29:17.999653 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-c699fc56-6339-4cb6-a746-73e1f528a42f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:29:18.097118 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c699fc56-6339-4cb6-a746-73e1f528a42f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:29:18.098454 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c699fc56-6339-4cb6-a746-73e1f528a42f service cinder] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:29:18.101867 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:29:18.116361 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 584/1182] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:29:17 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:29:18.162340 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8aad45-9a3b-4e72-9e70-c13e62ddbe4d None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:29:18.162623 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8aad45-9a3b-4e72-9e70-c13e62ddbe4d None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:29:18.162892 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8aad45-9a3b-4e72-9e70-c13e62ddbe4d None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:29:18.164225 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 599/1183] 10.222.0.15 () {52 vars in 830 bytes} [Mon Oct 31 12:29:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:29:18.167705 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c699fc56-6339-4cb6-a746-73e1f528a42f service cinder] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:29:18.168905 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-c699fc56-6339-4cb6-a746-73e1f528a42f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:29:18.268634 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c699fc56-6339-4cb6-a746-73e1f528a42f service cinder] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:29:18.364777 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6b6802-3a61-4082-ac6c-84b34b32eee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:29:18.365406 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6b6802-3a61-4082-ac6c-84b34b32eee2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:29:18.365587 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6b6802-3a61-4082-ac6c-84b34b32eee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:29:18.365857 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6b6802-3a61-4082-ac6c-84b34b32eee2 admin admin] PATH_INFO: `/v3/users/cd86e149510b4a1eb7d267a9a6a418ba` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:29:18.379217 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6b6802-3a61-4082-ac6c-84b34b32eee2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd86e149510b4a1eb7d267a9a6a418ba)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:29:18.380237 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: WARNING py.warnings [None req-ea6b6802-3a61-4082-ac6c-84b34b32eee2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 31 12:29:18.380237 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: warnings.warn(msg) Oct 31 12:29:18.380237 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: Oct 31 12:29:18.381444 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6b6802-3a61-4082-ac6c-84b34b32eee2 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:29:18.391903 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:29:18.450569 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-ea6b6802-3a61-4082-ac6c-84b34b32eee2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd86e149510b4a1eb7d267a9a6a418ba'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:29:18.460372 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-ea6b6802-3a61-4082-ac6c-84b34b32eee2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd86e149510b4a1eb7d267a9a6a418ba'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:29:18.460877 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-ea6b6802-3a61-4082-ac6c-84b34b32eee2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd86e149510b4a1eb7d267a9a6a418ba'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:29:18.483896 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-ea6b6802-3a61-4082-ac6c-84b34b32eee2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd86e149510b4a1eb7d267a9a6a418ba'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:29:18.497405 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-ea6b6802-3a61-4082-ac6c-84b34b32eee2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd86e149510b4a1eb7d267a9a6a418ba'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:29:18.503840 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 585/1184] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:29:18 2022] DELETE /identity/v3/users/cd86e149510b4a1eb7d267a9a6a418ba => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:29:18.508202 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37909d74-9a92-4e00-be85-0dba4c69029b None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:29:18.508653 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37909d74-9a92-4e00-be85-0dba4c69029b None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:29:18.508767 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37909d74-9a92-4e00-be85-0dba4c69029b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:29:18.565669 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-37909d74-9a92-4e00-be85-0dba4c69029b None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:29:18.566858 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-37909d74-9a92-4e00-be85-0dba4c69029b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:29:18.641371 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:29:18.653582 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 600/1185] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:29:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:29:18.663776 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6b6802-3a61-4082-ac6c-84b34b32eee2 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:29:18.665034 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-ea6b6802-3a61-4082-ac6c-84b34b32eee2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:29:18.764736 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6b6802-3a61-4082-ac6c-84b34b32eee2 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:29:18.818872 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:29:18.839437 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9361ff0b-cab1-4bd6-ad41-497dd86544d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:29:18.840041 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9361ff0b-cab1-4bd6-ad41-497dd86544d4 service neutron] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:29:18.840315 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9361ff0b-cab1-4bd6-ad41-497dd86544d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:29:18.840583 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9361ff0b-cab1-4bd6-ad41-497dd86544d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:29:18.842129 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-9361ff0b-cab1-4bd6-ad41-497dd86544d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:29:18.948885 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9361ff0b-cab1-4bd6-ad41-497dd86544d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:29:18.950216 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9361ff0b-cab1-4bd6-ad41-497dd86544d4 service neutron] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:29:18.968315 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 586/1186] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:29:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:29:19.334664 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37909d74-9a92-4e00-be85-0dba4c69029b None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:29:19.335949 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-37909d74-9a92-4e00-be85-0dba4c69029b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:29:19.347689 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37909d74-9a92-4e00-be85-0dba4c69029b None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:29:19.443918 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3cd486-b90f-41f2-a6b8-13771228388b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:29:19.444546 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3cd486-b90f-41f2-a6b8-13771228388b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:29:19.444817 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3cd486-b90f-41f2-a6b8-13771228388b admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:29:19.445082 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3cd486-b90f-41f2-a6b8-13771228388b admin admin] PATH_INFO: `/v3/projects/583d58e27a3d4577b832d7a0b58f3b11` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:29:19.455331 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3cd486-b90f-41f2-a6b8-13771228388b admin admin] RBAC: Authorizing `identity:delete_project(project_id=583d58e27a3d4577b832d7a0b58f3b11)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:29:19.456334 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: WARNING py.warnings [None req-ab3cd486-b90f-41f2-a6b8-13771228388b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 31 12:29:19.456334 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: warnings.warn(msg) Oct 31 12:29:19.456334 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: Oct 31 12:29:19.457571 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3cd486-b90f-41f2-a6b8-13771228388b admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:29:19.513610 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-ab3cd486-b90f-41f2-a6b8-13771228388b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '583d58e27a3d4577b832d7a0b58f3b11', 'request_id': 'req-ab3cd486-b90f-41f2-a6b8-13771228388b'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:29:19.527221 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-ab3cd486-b90f-41f2-a6b8-13771228388b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '583d58e27a3d4577b832d7a0b58f3b11', 'request_id': 'req-ab3cd486-b90f-41f2-a6b8-13771228388b'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:29:19.536292 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-ab3cd486-b90f-41f2-a6b8-13771228388b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '583d58e27a3d4577b832d7a0b58f3b11', 'request_id': 'req-ab3cd486-b90f-41f2-a6b8-13771228388b'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:29:19.560547 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-ab3cd486-b90f-41f2-a6b8-13771228388b admin admin] The token cache is being invalidate because project 583d58e27a3d4577b832d7a0b58f3b11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:29:19.560833 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-ab3cd486-b90f-41f2-a6b8-13771228388b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:29:19.561349 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-ab3cd486-b90f-41f2-a6b8-13771228388b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:29:19.562705 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 601/1187] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:29:19 2022] DELETE /identity/v3/projects/583d58e27a3d4577b832d7a0b58f3b11 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:29:28.966063 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=62 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:29:29.558544 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:30:45.639648 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bddd8d-9cae-43c1-9d97-095de75a1629 None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:30:45.640193 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bddd8d-9cae-43c1-9d97-095de75a1629 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:30:45.640193 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bddd8d-9cae-43c1-9d97-095de75a1629 None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:30:45.641598 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 587/1188] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 31 12:30:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:30:45.646689 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f216bc-16f4-4a08-b7ff-9a803631fe73 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:30:45.647120 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f216bc-16f4-4a08-b7ff-9a803631fe73 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:30:45.647293 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f216bc-16f4-4a08-b7ff-9a803631fe73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:30:45.725144 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-20f216bc-16f4-4a08-b7ff-9a803631fe73 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:30:45.726352 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-20f216bc-16f4-4a08-b7ff-9a803631fe73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:30:45.831959 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 602/1189] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 31 12:30:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:30:45.986606 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bddd8d-9cae-43c1-9d97-095de75a1629 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:30:45.988676 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-22bddd8d-9cae-43c1-9d97-095de75a1629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:30:46.113151 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bddd8d-9cae-43c1-9d97-095de75a1629 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:30:46.118561 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dcd859-1cf4-48ec-9e9b-ac83b68b22e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99889b1dab0b471d9c77887cffefc652', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:30:46.119184 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dcd859-1cf4-48ec-9e9b-ac83b68b22e4 service nova] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:30:46.119471 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dcd859-1cf4-48ec-9e9b-ac83b68b22e4 service nova] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:30:46.119740 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dcd859-1cf4-48ec-9e9b-ac83b68b22e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:30:46.128322 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dcd859-1cf4-48ec-9e9b-ac83b68b22e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:30:46.129921 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dcd859-1cf4-48ec-9e9b-ac83b68b22e4 service nova] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:30:46.139221 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 588/1190] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:30:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:30:47.213882 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659db954-45c2-48c9-b587-2da746f70daa None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:30:47.214429 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659db954-45c2-48c9-b587-2da746f70daa None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:30:47.214429 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659db954-45c2-48c9-b587-2da746f70daa None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:30:47.215811 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 603/1191] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 31 12:30:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:30:47.227276 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cca8a7-642c-47eb-ba72-f78e72d2e72e None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:30:47.227736 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cca8a7-642c-47eb-ba72-f78e72d2e72e None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:30:47.227955 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cca8a7-642c-47eb-ba72-f78e72d2e72e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:30:47.248131 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-53cca8a7-642c-47eb-ba72-f78e72d2e72e None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:30:47.249391 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-53cca8a7-642c-47eb-ba72-f78e72d2e72e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:30:47.262418 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 589/1192] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 31 12:30:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:30:47.415348 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659db954-45c2-48c9-b587-2da746f70daa None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:30:47.429634 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659db954-45c2-48c9-b587-2da746f70daa None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:30:47.435099 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70894b1c-9f27-4943-a56e-394c51b458e5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99889b1dab0b471d9c77887cffefc652', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:30:47.435721 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70894b1c-9f27-4943-a56e-394c51b458e5 service nova] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:30:47.435998 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70894b1c-9f27-4943-a56e-394c51b458e5 service nova] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:30:47.436264 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70894b1c-9f27-4943-a56e-394c51b458e5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:30:47.444682 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70894b1c-9f27-4943-a56e-394c51b458e5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:30:47.446241 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70894b1c-9f27-4943-a56e-394c51b458e5 service nova] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:30:47.456035 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 604/1193] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:30:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:30:47.868803 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9eef84-6c6f-4cea-93df-2aa5fb0f1941 None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:30:47.869069 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9eef84-6c6f-4cea-93df-2aa5fb0f1941 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:30:47.869346 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9eef84-6c6f-4cea-93df-2aa5fb0f1941 None None] PATH_INFO: `/v3` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:30:47.870637 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 590/1194] 10.222.0.20 () {54 vars in 925 bytes} [Mon Oct 31 12:30:47 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:30:47.878462 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274b44ee-3983-4c0d-bf75-f6f3d47f5367 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:30:47.878747 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274b44ee-3983-4c0d-bf75-f6f3d47f5367 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:30:47.879025 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274b44ee-3983-4c0d-bf75-f6f3d47f5367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:30:47.936849 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-274b44ee-3983-4c0d-bf75-f6f3d47f5367 None None] MFA Rules not processed for user `9e7777495d7a406c8c50b1ad52e47263`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:30:47.938025 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-274b44ee-3983-4c0d-bf75-f6f3d47f5367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:30:47.955572 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:30:48.016908 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 605/1195] 10.222.0.20 () {58 vars in 1015 bytes} [Mon Oct 31 12:30:47 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 31 12:30:51.629075 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74135550-2275-49ff-9610-809cdb8904b9 None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:30:51.629679 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74135550-2275-49ff-9610-809cdb8904b9 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:30:51.629679 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74135550-2275-49ff-9610-809cdb8904b9 None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:30:51.631055 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 591/1196] 10.222.0.15 () {54 vars in 961 bytes} [Mon Oct 31 12:30:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:30:51.636389 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a1c286-a61d-4333-8cd5-5014c473b1bf None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:30:51.636802 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a1c286-a61d-4333-8cd5-5014c473b1bf None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:30:51.636939 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a1c286-a61d-4333-8cd5-5014c473b1bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:30:51.657048 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-06a1c286-a61d-4333-8cd5-5014c473b1bf None None] MFA Rules not processed for user `9e7777495d7a406c8c50b1ad52e47263`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:30:51.658237 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-06a1c286-a61d-4333-8cd5-5014c473b1bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:30:51.666834 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 606/1197] 10.222.0.15 () {58 vars in 1058 bytes} [Mon Oct 31 12:30:51 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 31 12:30:51.674464 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74135550-2275-49ff-9610-809cdb8904b9 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:30:51.687610 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74135550-2275-49ff-9610-809cdb8904b9 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:30:51.693354 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d3e039-5554-46e9-8cb4-7222b8a51476 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e7777495d7a406c8c50b1ad52e47263', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:30:51.693947 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d3e039-5554-46e9-8cb4-7222b8a51476 service placement] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:30:51.694228 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d3e039-5554-46e9-8cb4-7222b8a51476 service placement] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:30:51.694498 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d3e039-5554-46e9-8cb4-7222b8a51476 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:30:51.702536 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d3e039-5554-46e9-8cb4-7222b8a51476 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:30:51.704732 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d3e039-5554-46e9-8cb4-7222b8a51476 service placement] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:30:51.714443 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 592/1198] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:30:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:30:58.918678 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ac4041-d874-49e5-bd27-6b80b02266dd None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:30:58.919324 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ac4041-d874-49e5-bd27-6b80b02266dd None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:30:58.919324 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ac4041-d874-49e5-bd27-6b80b02266dd None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:30:58.920600 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 607/1199] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 31 12:30:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:30:58.926055 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258bcb65-a2cb-4734-aa50-1312ec9d06d1 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:30:58.926467 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258bcb65-a2cb-4734-aa50-1312ec9d06d1 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:30:58.926606 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258bcb65-a2cb-4734-aa50-1312ec9d06d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:30:58.945728 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-258bcb65-a2cb-4734-aa50-1312ec9d06d1 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:30:58.946911 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-258bcb65-a2cb-4734-aa50-1312ec9d06d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:30:58.954805 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 593/1200] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 31 12:30:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:30:59.113428 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ac4041-d874-49e5-bd27-6b80b02266dd None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:30:59.127557 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ac4041-d874-49e5-bd27-6b80b02266dd None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:30:59.132907 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8528fdf-10e8-4ae0-8d52-8eef0c0fca28 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99889b1dab0b471d9c77887cffefc652', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:30:59.133526 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8528fdf-10e8-4ae0-8d52-8eef0c0fca28 service nova] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:30:59.133801 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8528fdf-10e8-4ae0-8d52-8eef0c0fca28 service nova] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:30:59.134069 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8528fdf-10e8-4ae0-8d52-8eef0c0fca28 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:30:59.141907 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8528fdf-10e8-4ae0-8d52-8eef0c0fca28 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:30:59.143541 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8528fdf-10e8-4ae0-8d52-8eef0c0fca28 service nova] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:30:59.153004 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 608/1201] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:30:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:31:00.219304 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406df3f6-5ac8-4107-9b4f-79d485a95500 None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:31:00.220445 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406df3f6-5ac8-4107-9b4f-79d485a95500 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:31:00.220569 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406df3f6-5ac8-4107-9b4f-79d485a95500 None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:31:00.221878 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 594/1202] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 31 12:31:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:31:00.226756 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3bdb4f-4166-4b18-af11-30c85d584efe None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:31:00.227199 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3bdb4f-4166-4b18-af11-30c85d584efe None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:31:00.227312 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3bdb4f-4166-4b18-af11-30c85d584efe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:31:00.247155 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-df3bdb4f-4166-4b18-af11-30c85d584efe None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:31:00.248393 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-df3bdb4f-4166-4b18-af11-30c85d584efe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:31:00.256865 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 609/1203] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 31 12:31:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 31 12:31:00.409668 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406df3f6-5ac8-4107-9b4f-79d485a95500 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:31:00.424036 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406df3f6-5ac8-4107-9b4f-79d485a95500 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:31:00.429543 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5533339c-d199-4f15-b197-4cf788db81c6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99889b1dab0b471d9c77887cffefc652', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:31:00.430141 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5533339c-d199-4f15-b197-4cf788db81c6 service nova] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:31:00.430417 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5533339c-d199-4f15-b197-4cf788db81c6 service nova] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:31:00.430686 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5533339c-d199-4f15-b197-4cf788db81c6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:31:00.440127 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5533339c-d199-4f15-b197-4cf788db81c6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:31:00.441924 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5533339c-d199-4f15-b197-4cf788db81c6 service nova] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:31:00.451790 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 595/1204] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:31:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:31:04.490933 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b99cf1-1744-444c-b8eb-14852534499f None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:31:04.491502 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b99cf1-1744-444c-b8eb-14852534499f None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:31:04.491502 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b99cf1-1744-444c-b8eb-14852534499f None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:31:04.492759 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 610/1205] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 31 12:31:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:31:04.498068 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0cb494-d0d2-47ec-a6f5-f222389a9ea0 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:31:04.498474 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0cb494-d0d2-47ec-a6f5-f222389a9ea0 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:31:04.498615 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0cb494-d0d2-47ec-a6f5-f222389a9ea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:31:04.518450 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-ee0cb494-d0d2-47ec-a6f5-f222389a9ea0 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:31:04.519664 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-ee0cb494-d0d2-47ec-a6f5-f222389a9ea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:31:04.527672 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 596/1206] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 31 12:31:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 31 12:31:04.815717 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b99cf1-1744-444c-b8eb-14852534499f None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:31:04.817401 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-93b99cf1-1744-444c-b8eb-14852534499f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:31:04.898697 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b99cf1-1744-444c-b8eb-14852534499f None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:31:04.902278 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e29d101-ca54-4217-a9bf-9d8e298b7594 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:31:04.902890 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e29d101-ca54-4217-a9bf-9d8e298b7594 service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:31:04.903191 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e29d101-ca54-4217-a9bf-9d8e298b7594 service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:31:04.903473 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e29d101-ca54-4217-a9bf-9d8e298b7594 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:31:04.913184 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e29d101-ca54-4217-a9bf-9d8e298b7594 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:31:04.914605 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e29d101-ca54-4217-a9bf-9d8e298b7594 service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:31:04.924600 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 611/1207] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:31:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:31:07.127384 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d016eb28-dc4f-41fb-9866-7d733acddb7e None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:31:07.127956 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d016eb28-dc4f-41fb-9866-7d733acddb7e None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:31:07.127956 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d016eb28-dc4f-41fb-9866-7d733acddb7e None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:31:07.129353 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 597/1208] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 31 12:31:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:31:07.134737 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6b64b0-35fd-45c7-9c9a-3c11180e1026 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:31:07.135226 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6b64b0-35fd-45c7-9c9a-3c11180e1026 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:31:07.135347 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6b64b0-35fd-45c7-9c9a-3c11180e1026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:31:07.156316 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-5e6b64b0-35fd-45c7-9c9a-3c11180e1026 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:31:07.157702 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-5e6b64b0-35fd-45c7-9c9a-3c11180e1026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:31:07.165431 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 612/1209] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 31 12:31:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:31:07.453889 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d016eb28-dc4f-41fb-9866-7d733acddb7e None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:31:07.465192 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d016eb28-dc4f-41fb-9866-7d733acddb7e None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:31:07.468621 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0054ad82-5c16-41c4-8dbb-e13259e265f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:31:07.469574 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0054ad82-5c16-41c4-8dbb-e13259e265f6 service neutron] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:31:07.469872 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0054ad82-5c16-41c4-8dbb-e13259e265f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:31:07.470140 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0054ad82-5c16-41c4-8dbb-e13259e265f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:31:07.478806 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0054ad82-5c16-41c4-8dbb-e13259e265f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:31:07.480245 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0054ad82-5c16-41c4-8dbb-e13259e265f6 service neutron] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:31:07.489970 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 598/1210] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:31:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:31:11.054145 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca0cfd5-3466-476c-9ee0-f5867a2a86b4 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:31:11.054627 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca0cfd5-3466-476c-9ee0-f5867a2a86b4 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:31:11.054722 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca0cfd5-3466-476c-9ee0-f5867a2a86b4 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:31:11.056170 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 613/1211] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 31 12:31:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:31:11.061448 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de360397-7a68-4469-97ba-3333680a1a58 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:31:11.061850 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de360397-7a68-4469-97ba-3333680a1a58 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:31:11.061991 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de360397-7a68-4469-97ba-3333680a1a58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:31:11.069622 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=81 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:31:11.082383 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-de360397-7a68-4469-97ba-3333680a1a58 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:31:11.083603 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-de360397-7a68-4469-97ba-3333680a1a58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:31:11.091577 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 599/1212] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 31 12:31:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:31:11.397567 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca0cfd5-3466-476c-9ee0-f5867a2a86b4 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:31:11.399297 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-8ca0cfd5-3466-476c-9ee0-f5867a2a86b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:31:11.462147 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:31:11.476099 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca0cfd5-3466-476c-9ee0-f5867a2a86b4 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:31:11.479705 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2a461e-7252-488b-a3f8-b263ba5b437a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d6d7ea077d49e8a66c6c6762499702', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:31:11.480424 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2a461e-7252-488b-a3f8-b263ba5b437a service cinder] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:31:11.480705 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2a461e-7252-488b-a3f8-b263ba5b437a service cinder] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:31:11.480975 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2a461e-7252-488b-a3f8-b263ba5b437a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:31:11.490006 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2a461e-7252-488b-a3f8-b263ba5b437a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:31:11.491450 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2a461e-7252-488b-a3f8-b263ba5b437a service cinder] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:31:11.500769 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 614/1213] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:31:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:31:13.727755 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c0d01f-7d0c-4ca3-8a9a-244c986c4749 None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:31:13.728335 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c0d01f-7d0c-4ca3-8a9a-244c986c4749 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:31:13.728335 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c0d01f-7d0c-4ca3-8a9a-244c986c4749 None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:31:13.729645 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 600/1214] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 31 12:31:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:31:13.736009 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f934118a-b3bd-4f00-9104-e270b60fc120 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:31:13.736497 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f934118a-b3bd-4f00-9104-e270b60fc120 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:31:13.736590 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f934118a-b3bd-4f00-9104-e270b60fc120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:31:13.756597 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-f934118a-b3bd-4f00-9104-e270b60fc120 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:31:13.757788 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-f934118a-b3bd-4f00-9104-e270b60fc120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:31:13.765579 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 615/1215] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 31 12:31:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:31:14.065186 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c0d01f-7d0c-4ca3-8a9a-244c986c4749 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:31:14.066827 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-d0c0d01f-7d0c-4ca3-8a9a-244c986c4749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:31:14.080046 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c0d01f-7d0c-4ca3-8a9a-244c986c4749 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:31:14.083522 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0cbe73-f013-47e3-a4fa-f1885c07066e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d6d7ea077d49e8a66c6c6762499702', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:31:14.084105 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0cbe73-f013-47e3-a4fa-f1885c07066e service cinder] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:31:14.084379 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0cbe73-f013-47e3-a4fa-f1885c07066e service cinder] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:31:14.084651 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0cbe73-f013-47e3-a4fa-f1885c07066e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:31:14.093220 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0cbe73-f013-47e3-a4fa-f1885c07066e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:31:14.095435 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0cbe73-f013-47e3-a4fa-f1885c07066e service cinder] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:31:14.104576 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 601/1216] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:31:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:31:23.751548 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=11 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:31:24.100464 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=8 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:32:38.841519 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ddccb1-f1a3-4e32-99bb-3347dc3c21fd None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:38.842568 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ddccb1-f1a3-4e32-99bb-3347dc3c21fd None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:38.843052 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ddccb1-f1a3-4e32-99bb-3347dc3c21fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:38.881357 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-89ddccb1-f1a3-4e32-99bb-3347dc3c21fd None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:32:38.882560 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-89ddccb1-f1a3-4e32-99bb-3347dc3c21fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:32:38.893011 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 616/1217] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:32:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 31 12:32:38.904295 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5481b221-0188-4de9-8c97-e2865a14d9fe None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:38.905305 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5481b221-0188-4de9-8c97-e2865a14d9fe None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:38.905785 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5481b221-0188-4de9-8c97-e2865a14d9fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:38.937111 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-5481b221-0188-4de9-8c97-e2865a14d9fe None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:32:38.938817 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-5481b221-0188-4de9-8c97-e2865a14d9fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:32:38.950857 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 602/1218] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:32:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:32:38.955664 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5481b221-0188-4de9-8c97-e2865a14d9fe None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:38.956176 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f760e66b-5bdc-4cab-879f-ee617ce72f4a None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:38.960655 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f760e66b-5bdc-4cab-879f-ee617ce72f4a None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:38.961271 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f760e66b-5bdc-4cab-879f-ee617ce72f4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:38.968723 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5481b221-0188-4de9-8c97-e2865a14d9fe None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:38.987966 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ecedb7-87d0-42ba-9cf9-a75ada7623ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:38.989138 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ecedb7-87d0-42ba-9cf9-a75ada7623ff admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:38.989423 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ecedb7-87d0-42ba-9cf9-a75ada7623ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:38.989699 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ecedb7-87d0-42ba-9cf9-a75ada7623ff admin admin] PATH_INFO: `/v3/domains` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:38.990360 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ecedb7-87d0-42ba-9cf9-a75ada7623ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:32:38.990679 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ecedb7-87d0-42ba-9cf9-a75ada7623ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:38.991735 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: WARNING py.warnings [None req-40ecedb7-87d0-42ba-9cf9-a75ada7623ff admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 31 12:32:38.991735 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: warnings.warn(msg) Oct 31 12:32:38.991735 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: Oct 31 12:32:38.992986 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ecedb7-87d0-42ba-9cf9-a75ada7623ff admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:38.996337 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-f760e66b-5bdc-4cab-879f-ee617ce72f4a None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:32:38.997774 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-f760e66b-5bdc-4cab-879f-ee617ce72f4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:32:39.006294 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 603/1219] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Oct 31 12:32:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:32:39.010194 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d38e40-c90e-4fbb-8b89-80265712a1af None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:39.010683 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d38e40-c90e-4fbb-8b89-80265712a1af None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:39.011122 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d38e40-c90e-4fbb-8b89-80265712a1af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:39.013618 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 617/1220] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:32:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:32:39.014688 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f760e66b-5bdc-4cab-879f-ee617ce72f4a None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:39.039140 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f760e66b-5bdc-4cab-879f-ee617ce72f4a None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:39.052609 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-92d38e40-c90e-4fbb-8b89-80265712a1af None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:32:39.053984 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-92d38e40-c90e-4fbb-8b89-80265712a1af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:32:39.054483 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e4ca00-8dda-41fb-92c8-39327255a11d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:39.055338 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e4ca00-8dda-41fb-92c8-39327255a11d admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:39.055787 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e4ca00-8dda-41fb-92c8-39327255a11d admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:39.056203 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e4ca00-8dda-41fb-92c8-39327255a11d admin admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:39.057442 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e4ca00-8dda-41fb-92c8-39327255a11d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:39.058625 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: WARNING py.warnings [None req-c0e4ca00-8dda-41fb-92c8-39327255a11d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 31 12:32:39.058625 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: warnings.warn(msg) Oct 31 12:32:39.058625 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: Oct 31 12:32:39.060331 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e4ca00-8dda-41fb-92c8-39327255a11d admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:39.081088 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 604/1221] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:32:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 31 12:32:39.082823 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8bf79b-b734-43ba-bbb0-041c64dd12da None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:39.083285 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8bf79b-b734-43ba-bbb0-041c64dd12da None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:39.083575 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8bf79b-b734-43ba-bbb0-041c64dd12da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:39.127445 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-cb8bf79b-b734-43ba-bbb0-041c64dd12da None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:32:39.131676 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-cb8bf79b-b734-43ba-bbb0-041c64dd12da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:32:39.137458 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 618/1222] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:32:39 2022] POST /identity/v3/projects => generated 385 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:32:39.139814 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301c115d-5989-48fc-9153-702ef6ab985a None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:39.142157 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301c115d-5989-48fc-9153-702ef6ab985a None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:39.142809 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301c115d-5989-48fc-9153-702ef6ab985a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:39.191721 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-301c115d-5989-48fc-9153-702ef6ab985a None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:32:39.195242 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-301c115d-5989-48fc-9153-702ef6ab985a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:32:39.255318 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 605/1223] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:32:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:32:39.256655 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8bf79b-b734-43ba-bbb0-041c64dd12da None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:39.268536 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8bf79b-b734-43ba-bbb0-041c64dd12da None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:39.274864 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175a3d7d-1175-43d6-b552-9a8fa8fe55e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:39.275946 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175a3d7d-1175-43d6-b552-9a8fa8fe55e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:39.276465 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175a3d7d-1175-43d6-b552-9a8fa8fe55e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:39.277445 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175a3d7d-1175-43d6-b552-9a8fa8fe55e4 admin admin] PATH_INFO: `/v3/users` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:39.278587 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175a3d7d-1175-43d6-b552-9a8fa8fe55e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:39.280103 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: WARNING py.warnings [None req-175a3d7d-1175-43d6-b552-9a8fa8fe55e4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 31 12:32:39.280103 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: warnings.warn(msg) Oct 31 12:32:39.280103 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: Oct 31 12:32:39.282394 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175a3d7d-1175-43d6-b552-9a8fa8fe55e4 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:39.304933 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 619/1224] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:32:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:32:39.307103 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7728510a-31f2-4ab9-a2b7-420c41e86702 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:39.307103 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7728510a-31f2-4ab9-a2b7-420c41e86702 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:39.307624 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7728510a-31f2-4ab9-a2b7-420c41e86702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:39.312452 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 606/1225] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:32:39 2022] POST /identity/v3/users => generated 293 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:32:39.313584 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02705648-be22-492b-8a2b-68c5ac68ebb5 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:39.313918 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02705648-be22-492b-8a2b-68c5ac68ebb5 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:39.314197 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02705648-be22-492b-8a2b-68c5ac68ebb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:39.336264 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-02705648-be22-492b-8a2b-68c5ac68ebb5 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:32:39.337470 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-02705648-be22-492b-8a2b-68c5ac68ebb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:32:39.339266 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-7728510a-31f2-4ab9-a2b7-420c41e86702 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:32:39.340612 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-7728510a-31f2-4ab9-a2b7-420c41e86702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:32:39.345710 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 607/1226] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:32:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:32:39.346900 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a16ac0-2b18-4ba9-9101-b8062cdb7d45 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:39.347245 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a16ac0-2b18-4ba9-9101-b8062cdb7d45 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:39.348121 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a16ac0-2b18-4ba9-9101-b8062cdb7d45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:39.350350 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 620/1227] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:32:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:32:39.352324 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7728510a-31f2-4ab9-a2b7-420c41e86702 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:39.363481 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7728510a-31f2-4ab9-a2b7-420c41e86702 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:39.368060 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-03a16ac0-2b18-4ba9-9101-b8062cdb7d45 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:32:39.368213 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc4269d-8fe7-4682-b4e0-e958814de6d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:39.369061 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc4269d-8fe7-4682-b4e0-e958814de6d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:39.369253 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-03a16ac0-2b18-4ba9-9101-b8062cdb7d45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:32:39.369363 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc4269d-8fe7-4682-b4e0-e958814de6d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:39.369599 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc4269d-8fe7-4682-b4e0-e958814de6d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:39.370269 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc4269d-8fe7-4682-b4e0-e958814de6d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:32:39.370569 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc4269d-8fe7-4682-b4e0-e958814de6d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:39.372182 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc4269d-8fe7-4682-b4e0-e958814de6d9 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:39.377256 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 608/1228] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:32:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:32:39.379864 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d72b9f-75ab-48eb-b237-eca554beddff None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:39.380221 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d72b9f-75ab-48eb-b237-eca554beddff None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:39.380507 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d72b9f-75ab-48eb-b237-eca554beddff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:39.387206 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 621/1229] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Oct 31 12:32:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:32:39.387206 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc4269d-8fe7-4682-b4e0-e958814de6d9 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:39.398062 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc4269d-8fe7-4682-b4e0-e958814de6d9 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:39.402969 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47502948-63f5-4ca1-ad0e-c2c96cef729a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:39.403614 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47502948-63f5-4ca1-ad0e-c2c96cef729a admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:39.403899 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47502948-63f5-4ca1-ad0e-c2c96cef729a admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:39.404161 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-32d72b9f-75ab-48eb-b237-eca554beddff None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:32:39.404276 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47502948-63f5-4ca1-ad0e-c2c96cef729a admin admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:39.404798 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47502948-63f5-4ca1-ad0e-c2c96cef729a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:32:39.405137 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47502948-63f5-4ca1-ad0e-c2c96cef729a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:39.405360 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-32d72b9f-75ab-48eb-b237-eca554beddff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:32:39.406707 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47502948-63f5-4ca1-ad0e-c2c96cef729a admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:39.413707 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 609/1230] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:32:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:32:39.415423 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d72b9f-75ab-48eb-b237-eca554beddff None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:39.420355 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 622/1231] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Oct 31 12:32:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:32:39.421581 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc7f552-d9c3-49bf-9cf1-fe0ab30f0243 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:39.421869 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc7f552-d9c3-49bf-9cf1-fe0ab30f0243 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:39.422140 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc7f552-d9c3-49bf-9cf1-fe0ab30f0243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:39.429692 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d72b9f-75ab-48eb-b237-eca554beddff None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:39.434261 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d3005f-1e6c-4dcc-a96c-194a83359a67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:39.434866 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d3005f-1e6c-4dcc-a96c-194a83359a67 admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:39.435143 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d3005f-1e6c-4dcc-a96c-194a83359a67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:39.435443 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d3005f-1e6c-4dcc-a96c-194a83359a67 admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:39.436083 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d3005f-1e6c-4dcc-a96c-194a83359a67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:39.437666 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d3005f-1e6c-4dcc-a96c-194a83359a67 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:39.443943 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-fcc7f552-d9c3-49bf-9cf1-fe0ab30f0243 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:32:39.445144 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-fcc7f552-d9c3-49bf-9cf1-fe0ab30f0243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:32:39.451768 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 610/1232] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:32:39 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:32:39.451768 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7179d36-b048-44f2-afd8-fb151a79e984 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:39.451768 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7179d36-b048-44f2-afd8-fb151a79e984 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:39.451768 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7179d36-b048-44f2-afd8-fb151a79e984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:39.453284 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 623/1233] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:32:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:32:39.456578 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f98a16-3490-47dd-92d6-e67b261a6cb8 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:39.457497 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f98a16-3490-47dd-92d6-e67b261a6cb8 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:39.457833 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f98a16-3490-47dd-92d6-e67b261a6cb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:39.470850 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-c7179d36-b048-44f2-afd8-fb151a79e984 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:32:39.472074 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-c7179d36-b048-44f2-afd8-fb151a79e984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:32:39.477175 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-36f98a16-3490-47dd-92d6-e67b261a6cb8 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:32:39.478350 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-36f98a16-3490-47dd-92d6-e67b261a6cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:32:39.480210 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 611/1234] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:32:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:32:39.481386 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e0f17a-0cac-43f5-8a2c-802b12dc0aa4 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:39.481666 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e0f17a-0cac-43f5-8a2c-802b12dc0aa4 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:39.481937 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e0f17a-0cac-43f5-8a2c-802b12dc0aa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:39.489367 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 624/1235] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:32:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:32:39.490041 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f98a16-3490-47dd-92d6-e67b261a6cb8 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:39.499953 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f98a16-3490-47dd-92d6-e67b261a6cb8 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:39.501992 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-52e0f17a-0cac-43f5-8a2c-802b12dc0aa4 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:32:39.503194 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-52e0f17a-0cac-43f5-8a2c-802b12dc0aa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:32:39.505250 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e444a1c-f09f-488f-8f57-ef131221a047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:39.505847 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e444a1c-f09f-488f-8f57-ef131221a047 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:39.506226 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e444a1c-f09f-488f-8f57-ef131221a047 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:39.506500 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e444a1c-f09f-488f-8f57-ef131221a047 admin admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:39.507114 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e444a1c-f09f-488f-8f57-ef131221a047 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:39.509184 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e444a1c-f09f-488f-8f57-ef131221a047 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:39.512376 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 612/1236] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:32:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:32:39.513631 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342a4b71-123a-49b5-9cb1-e5ba8d279f7b None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:39.513814 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342a4b71-123a-49b5-9cb1-e5ba8d279f7b None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:39.514087 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342a4b71-123a-49b5-9cb1-e5ba8d279f7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:39.534182 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-342a4b71-123a-49b5-9cb1-e5ba8d279f7b None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:32:39.536056 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-342a4b71-123a-49b5-9cb1-e5ba8d279f7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:32:39.541533 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 625/1237] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:32:39 2022] POST /identity/v3/projects => generated 379 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:32:39.541533 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e444a1c-f09f-488f-8f57-ef131221a047 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:39.590193 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:32:39.594515 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 613/1238] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:32:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:32:39.595432 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342a4b71-123a-49b5-9cb1-e5ba8d279f7b None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:39.604869 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342a4b71-123a-49b5-9cb1-e5ba8d279f7b None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:39.608133 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e444a1c-f09f-488f-8f57-ef131221a047 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:39.609447 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d314c1-fa5e-44ac-8eff-38f579b3ec65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:39.610036 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d314c1-fa5e-44ac-8eff-38f579b3ec65 admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:39.610313 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d314c1-fa5e-44ac-8eff-38f579b3ec65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:39.610584 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d314c1-fa5e-44ac-8eff-38f579b3ec65 admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:39.611244 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d314c1-fa5e-44ac-8eff-38f579b3ec65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:39.612850 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d314c1-fa5e-44ac-8eff-38f579b3ec65 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:39.613515 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f6f6bf-faaa-4f7b-a052-1006a6aeba34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:39.614695 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f6f6bf-faaa-4f7b-a052-1006a6aeba34 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:39.614978 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f6f6bf-faaa-4f7b-a052-1006a6aeba34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:39.615290 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f6f6bf-faaa-4f7b-a052-1006a6aeba34 admin admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:39.615915 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f6f6bf-faaa-4f7b-a052-1006a6aeba34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:39.617799 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f6f6bf-faaa-4f7b-a052-1006a6aeba34 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:39.626303 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 614/1239] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:32:39 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:32:39.626303 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74849d1a-51b8-492c-b63f-ec6a79af67e3 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:39.626303 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74849d1a-51b8-492c-b63f-ec6a79af67e3 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:39.626303 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74849d1a-51b8-492c-b63f-ec6a79af67e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:39.633209 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:32:39.648262 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-74849d1a-51b8-492c-b63f-ec6a79af67e3 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:32:39.648427 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 626/1240] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:32:39 2022] POST /identity/v3/projects => generated 393 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:32:39.649250 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f6f6bf-faaa-4f7b-a052-1006a6aeba34 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:39.649585 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-74849d1a-51b8-492c-b63f-ec6a79af67e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:32:39.710184 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 615/1241] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:32:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:32:39.711030 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74849d1a-51b8-492c-b63f-ec6a79af67e3 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:39.716353 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f6f6bf-faaa-4f7b-a052-1006a6aeba34 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:39.720296 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74849d1a-51b8-492c-b63f-ec6a79af67e3 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:39.721029 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6897b38a-499a-4770-b33a-92994fcf16f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:39.721646 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6897b38a-499a-4770-b33a-92994fcf16f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:39.721921 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6897b38a-499a-4770-b33a-92994fcf16f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:39.722217 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6897b38a-499a-4770-b33a-92994fcf16f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:39.722935 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6897b38a-499a-4770-b33a-92994fcf16f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:32:39.723309 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6897b38a-499a-4770-b33a-92994fcf16f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:39.724972 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6897b38a-499a-4770-b33a-92994fcf16f8 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:39.725534 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d2bfbd-f582-4a25-95e8-c9ffc7ac8ca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:39.726114 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d2bfbd-f582-4a25-95e8-c9ffc7ac8ca1 admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:39.726400 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d2bfbd-f582-4a25-95e8-c9ffc7ac8ca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:39.726673 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d2bfbd-f582-4a25-95e8-c9ffc7ac8ca1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:39.727342 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d2bfbd-f582-4a25-95e8-c9ffc7ac8ca1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:32:39.727646 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d2bfbd-f582-4a25-95e8-c9ffc7ac8ca1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:39.729235 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d2bfbd-f582-4a25-95e8-c9ffc7ac8ca1 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:39.736989 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 627/1242] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Oct 31 12:32:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:32:39.737800 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6897b38a-499a-4770-b33a-92994fcf16f8 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:39.748027 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6897b38a-499a-4770-b33a-92994fcf16f8 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:39.749220 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 616/1243] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Oct 31 12:32:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:32:39.750036 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d2bfbd-f582-4a25-95e8-c9ffc7ac8ca1 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:39.752711 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33f50e5-9c41-4b29-8526-e13e86c11093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:39.753343 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33f50e5-9c41-4b29-8526-e13e86c11093 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:39.753671 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33f50e5-9c41-4b29-8526-e13e86c11093 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:39.754109 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33f50e5-9c41-4b29-8526-e13e86c11093 admin admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:39.754723 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33f50e5-9c41-4b29-8526-e13e86c11093 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:32:39.755021 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33f50e5-9c41-4b29-8526-e13e86c11093 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:39.756644 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33f50e5-9c41-4b29-8526-e13e86c11093 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:39.765808 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d2bfbd-f582-4a25-95e8-c9ffc7ac8ca1 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:39.769079 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 628/1244] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Oct 31 12:32:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:32:39.771516 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33f50e5-9c41-4b29-8526-e13e86c11093 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:39.772822 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f4c448-843e-42e5-a951-c4eb1a898a11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:39.773684 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f4c448-843e-42e5-a951-c4eb1a898a11 admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:39.773962 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f4c448-843e-42e5-a951-c4eb1a898a11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:39.774228 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f4c448-843e-42e5-a951-c4eb1a898a11 admin admin] PATH_INFO: `/v3/users` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:39.774993 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f4c448-843e-42e5-a951-c4eb1a898a11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:39.777251 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f4c448-843e-42e5-a951-c4eb1a898a11 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:39.781300 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33f50e5-9c41-4b29-8526-e13e86c11093 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:39.786647 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151dd17d-1ab0-4e15-a509-09c82f57ff17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:39.787279 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151dd17d-1ab0-4e15-a509-09c82f57ff17 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:39.787564 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151dd17d-1ab0-4e15-a509-09c82f57ff17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:39.787999 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151dd17d-1ab0-4e15-a509-09c82f57ff17 admin admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:39.788718 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151dd17d-1ab0-4e15-a509-09c82f57ff17 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:32:39.788910 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151dd17d-1ab0-4e15-a509-09c82f57ff17 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:39.790596 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151dd17d-1ab0-4e15-a509-09c82f57ff17 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:39.802683 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 617/1245] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:32:39 2022] POST /identity/v3/users => generated 290 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:32:39.803877 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f4c448-843e-42e5-a951-c4eb1a898a11 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:39.804181 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 629/1246] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Oct 31 12:32:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:32:39.805028 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151dd17d-1ab0-4e15-a509-09c82f57ff17 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:39.815114 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151dd17d-1ab0-4e15-a509-09c82f57ff17 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:39.816885 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f4c448-843e-42e5-a951-c4eb1a898a11 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:39.821130 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f27bbc-b0cf-4ca9-8f31-2428359d55eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:39.821691 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9c71cb-81bf-407f-a74b-ce69c738e30a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:39.821807 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f27bbc-b0cf-4ca9-8f31-2428359d55eb admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:39.821999 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f27bbc-b0cf-4ca9-8f31-2428359d55eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:39.822288 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f27bbc-b0cf-4ca9-8f31-2428359d55eb admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:39.822463 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9c71cb-81bf-407f-a74b-ce69c738e30a admin admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:39.822891 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9c71cb-81bf-407f-a74b-ce69c738e30a admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:39.823035 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f27bbc-b0cf-4ca9-8f31-2428359d55eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:39.823236 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9c71cb-81bf-407f-a74b-ce69c738e30a admin admin] PATH_INFO: `/v3/projects/bfbc3a46f12942299b99cdd20c989e1f/users/b2c03f0365ee4a26b8c076ae365325b2/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:39.824960 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f27bbc-b0cf-4ca9-8f31-2428359d55eb admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:39.838647 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9c71cb-81bf-407f-a74b-ce69c738e30a admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfbc3a46f12942299b99cdd20c989e1f, user_id=b2c03f0365ee4a26b8c076ae365325b2, role_id=ca1a5695f8bd499aa0b8cedaeecd2cc6)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:39.841522 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9c71cb-81bf-407f-a74b-ce69c738e30a admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:39.849497 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 630/1247] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:32:39 2022] POST /identity/v3/users => generated 297 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:32:39.850187 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f27bbc-b0cf-4ca9-8f31-2428359d55eb admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:39.854762 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 618/1248] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:32:39 2022] PUT /identity/v3/projects/bfbc3a46f12942299b99cdd20c989e1f/users/b2c03f0365ee4a26b8c076ae365325b2/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:32:39.855580 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9c71cb-81bf-407f-a74b-ce69c738e30a admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:39.863556 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f27bbc-b0cf-4ca9-8f31-2428359d55eb admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:39.918199 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9c71cb-81bf-407f-a74b-ce69c738e30a admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:39.921050 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d947993-8c9d-4502-8a20-388c2ae94bd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:39.921667 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d947993-8c9d-4502-8a20-388c2ae94bd9 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:39.921947 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d947993-8c9d-4502-8a20-388c2ae94bd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:39.922219 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d947993-8c9d-4502-8a20-388c2ae94bd9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:39.922678 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7bd111-be86-4948-bc18-3fa046a90443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:39.922862 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d947993-8c9d-4502-8a20-388c2ae94bd9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:32:39.923186 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d947993-8c9d-4502-8a20-388c2ae94bd9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:39.923309 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7bd111-be86-4948-bc18-3fa046a90443 admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:39.923603 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7bd111-be86-4948-bc18-3fa046a90443 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:39.923876 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7bd111-be86-4948-bc18-3fa046a90443 admin admin] PATH_INFO: `/v3/projects` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:39.924502 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7bd111-be86-4948-bc18-3fa046a90443 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:39.924829 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d947993-8c9d-4502-8a20-388c2ae94bd9 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:39.926921 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7bd111-be86-4948-bc18-3fa046a90443 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:39.938203 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 631/1249] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Oct 31 12:32:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:32:39.945667 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d947993-8c9d-4502-8a20-388c2ae94bd9 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:39.954515 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d947993-8c9d-4502-8a20-388c2ae94bd9 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:39.956953 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 619/1250] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:32:39 2022] POST /identity/v3/projects => generated 369 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:32:39.957985 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7bd111-be86-4948-bc18-3fa046a90443 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:39.959210 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f78ea4-de06-45b4-992a-634df1aa11d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:39.959832 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f78ea4-de06-45b4-992a-634df1aa11d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:39.960198 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f78ea4-de06-45b4-992a-634df1aa11d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:39.960422 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f78ea4-de06-45b4-992a-634df1aa11d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:39.961073 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f78ea4-de06-45b4-992a-634df1aa11d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:39.962952 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f78ea4-de06-45b4-992a-634df1aa11d0 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:39.978858 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:32:39.992866 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 632/1251] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:32:39 2022] POST /identity/v3/projects => generated 375 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:32:39.993352 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:32:39.993678 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f78ea4-de06-45b4-992a-634df1aa11d0 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:40.027304 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7bd111-be86-4948-bc18-3fa046a90443 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:40.059142 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f78ea4-de06-45b4-992a-634df1aa11d0 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:40.063806 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1329e01b-cf9f-4c95-81c8-909d030624ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:40.064446 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1329e01b-cf9f-4c95-81c8-909d030624ab admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:40.064723 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1329e01b-cf9f-4c95-81c8-909d030624ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:40.065023 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1329e01b-cf9f-4c95-81c8-909d030624ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:40.065659 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1329e01b-cf9f-4c95-81c8-909d030624ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:40.067838 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1329e01b-cf9f-4c95-81c8-909d030624ab admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:40.077782 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 633/1252] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:32:39 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:32:40.078530 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1329e01b-cf9f-4c95-81c8-909d030624ab admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:40.087857 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f9c6b5-de55-4553-831d-d6d11a13024c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:40.088472 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f9c6b5-de55-4553-831d-d6d11a13024c admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:40.088758 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f9c6b5-de55-4553-831d-d6d11a13024c admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:40.089163 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f9c6b5-de55-4553-831d-d6d11a13024c admin admin] PATH_INFO: `/v3/projects` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:40.089795 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f9c6b5-de55-4553-831d-d6d11a13024c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:40.089941 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1329e01b-cf9f-4c95-81c8-909d030624ab admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:40.091722 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f9c6b5-de55-4553-831d-d6d11a13024c admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:40.094443 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7a6bbe-e927-498d-b684-d04b678ea4b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:40.095023 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7a6bbe-e927-498d-b684-d04b678ea4b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:40.095334 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7a6bbe-e927-498d-b684-d04b678ea4b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:40.095614 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7a6bbe-e927-498d-b684-d04b678ea4b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:40.096223 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7a6bbe-e927-498d-b684-d04b678ea4b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:40.098733 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7a6bbe-e927-498d-b684-d04b678ea4b3 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:40.123298 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 620/1253] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:32:39 2022] POST /identity/v3/projects => generated 387 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:32:40.126764 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 634/1254] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:32:40 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:32:40.128069 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b23abcc-9d72-420e-a1ce-244d77c43158 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:40.128497 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b23abcc-9d72-420e-a1ce-244d77c43158 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:40.129267 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b23abcc-9d72-420e-a1ce-244d77c43158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:40.131106 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f9c6b5-de55-4553-831d-d6d11a13024c admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:40.149389 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-7b23abcc-9d72-420e-a1ce-244d77c43158 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:32:40.150599 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-7b23abcc-9d72-420e-a1ce-244d77c43158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:32:40.194135 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f9c6b5-de55-4553-831d-d6d11a13024c admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:40.209591 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 635/1255] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:32:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:32:40.210637 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b23abcc-9d72-420e-a1ce-244d77c43158 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:40.219751 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b23abcc-9d72-420e-a1ce-244d77c43158 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:40.224766 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883d75e1-3105-4c98-8f0c-e4dc09a83d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:40.225372 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883d75e1-3105-4c98-8f0c-e4dc09a83d14 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:40.225647 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883d75e1-3105-4c98-8f0c-e4dc09a83d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:40.225921 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883d75e1-3105-4c98-8f0c-e4dc09a83d14 admin admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:40.226642 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883d75e1-3105-4c98-8f0c-e4dc09a83d14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:40.228468 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883d75e1-3105-4c98-8f0c-e4dc09a83d14 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:40.257395 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 636/1256] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:32:40 2022] POST /identity/v3/projects => generated 363 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:32:40.257582 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763815b7-9956-4db2-93fd-b630f259f44b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:40.257989 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763815b7-9956-4db2-93fd-b630f259f44b admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:40.258270 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883d75e1-3105-4c98-8f0c-e4dc09a83d14 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:40.258378 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763815b7-9956-4db2-93fd-b630f259f44b admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:40.258533 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763815b7-9956-4db2-93fd-b630f259f44b admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:40.259197 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763815b7-9956-4db2-93fd-b630f259f44b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:40.260803 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763815b7-9956-4db2-93fd-b630f259f44b admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:40.272716 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 621/1257] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:32:40 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:32:40.273359 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763815b7-9956-4db2-93fd-b630f259f44b admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:40.282764 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763815b7-9956-4db2-93fd-b630f259f44b admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:40.287964 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:32:40.314317 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:32:40.329784 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883d75e1-3105-4c98-8f0c-e4dc09a83d14 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:40.335327 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92095046-c089-491e-b421-20481ca83c70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:40.335935 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92095046-c089-491e-b421-20481ca83c70 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:40.336210 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92095046-c089-491e-b421-20481ca83c70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:40.336502 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92095046-c089-491e-b421-20481ca83c70 admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:40.337095 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92095046-c089-491e-b421-20481ca83c70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:40.339009 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92095046-c089-491e-b421-20481ca83c70 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:40.343016 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d733db4-51aa-4a09-b46c-aa012de30675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:40.343649 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d733db4-51aa-4a09-b46c-aa012de30675 admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:40.343925 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d733db4-51aa-4a09-b46c-aa012de30675 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:40.344197 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d733db4-51aa-4a09-b46c-aa012de30675 admin admin] PATH_INFO: `/v3/users` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:40.344841 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d733db4-51aa-4a09-b46c-aa012de30675 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:40.346732 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d733db4-51aa-4a09-b46c-aa012de30675 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:40.365734 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 637/1258] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:32:40 2022] POST /identity/v3/users => generated 285 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:32:40.365734 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92095046-c089-491e-b421-20481ca83c70 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:40.371468 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 622/1259] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:32:40 2022] POST /identity/v3/users => generated 288 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:32:40.372490 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d733db4-51aa-4a09-b46c-aa012de30675 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:40.377237 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92095046-c089-491e-b421-20481ca83c70 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:40.383264 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48163410-5ea1-4907-9650-11f2eaf831d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:40.383481 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d733db4-51aa-4a09-b46c-aa012de30675 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:40.383895 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48163410-5ea1-4907-9650-11f2eaf831d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:40.384175 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48163410-5ea1-4907-9650-11f2eaf831d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:40.384442 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48163410-5ea1-4907-9650-11f2eaf831d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:40.385081 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48163410-5ea1-4907-9650-11f2eaf831d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:40.386660 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48163410-5ea1-4907-9650-11f2eaf831d4 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:40.388115 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35218b4-47f7-4471-a6ce-2f3f3b2ed4f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:40.388697 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35218b4-47f7-4471-a6ce-2f3f3b2ed4f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:40.388968 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35218b4-47f7-4471-a6ce-2f3f3b2ed4f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:40.389236 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35218b4-47f7-4471-a6ce-2f3f3b2ed4f3 admin admin] PATH_INFO: `/v3/users` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:40.389877 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35218b4-47f7-4471-a6ce-2f3f3b2ed4f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:40.391793 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35218b4-47f7-4471-a6ce-2f3f3b2ed4f3 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:40.397304 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 638/1260] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:32:40 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:32:40.398167 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48163410-5ea1-4907-9650-11f2eaf831d4 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:40.408727 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48163410-5ea1-4907-9650-11f2eaf831d4 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:40.413681 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c198ec-82c0-456b-97de-4fdbd769acf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:40.414279 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c198ec-82c0-456b-97de-4fdbd769acf6 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:40.414556 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c198ec-82c0-456b-97de-4fdbd769acf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:40.414828 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c198ec-82c0-456b-97de-4fdbd769acf6 admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:40.415482 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c198ec-82c0-456b-97de-4fdbd769acf6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:40.417283 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 623/1261] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:32:40 2022] POST /identity/v3/users => generated 294 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:32:40.417485 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c198ec-82c0-456b-97de-4fdbd769acf6 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:40.417964 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35218b4-47f7-4471-a6ce-2f3f3b2ed4f3 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:40.440908 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 639/1262] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:32:40 2022] POST /identity/v3/users => generated 287 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:32:40.441978 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c198ec-82c0-456b-97de-4fdbd769acf6 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:40.451308 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c198ec-82c0-456b-97de-4fdbd769acf6 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:40.455987 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c0daba-b46b-4d40-80ab-776e85eb988f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:40.456575 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c0daba-b46b-4d40-80ab-776e85eb988f admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:40.457367 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c0daba-b46b-4d40-80ab-776e85eb988f admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:40.457633 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c0daba-b46b-4d40-80ab-776e85eb988f admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:40.458252 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c0daba-b46b-4d40-80ab-776e85eb988f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:40.460166 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c0daba-b46b-4d40-80ab-776e85eb988f admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:40.474161 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35218b4-47f7-4471-a6ce-2f3f3b2ed4f3 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:40.478142 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a9a8fc-6fce-404c-8064-aa610045432e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:40.478804 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a9a8fc-6fce-404c-8064-aa610045432e service neutron] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:40.479080 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a9a8fc-6fce-404c-8064-aa610045432e service neutron] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:40.479380 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a9a8fc-6fce-404c-8064-aa610045432e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:40.481686 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 640/1263] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:32:40 2022] POST /identity/v3/users => generated 282 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:32:40.482566 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c0daba-b46b-4d40-80ab-776e85eb988f admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:40.491331 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a9a8fc-6fce-404c-8064-aa610045432e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:40.492669 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c0daba-b46b-4d40-80ab-776e85eb988f admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:40.492816 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a9a8fc-6fce-404c-8064-aa610045432e service neutron] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:40.497255 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f022b1-185e-40b3-aa99-b3c0a68e2670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:40.497838 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f022b1-185e-40b3-aa99-b3c0a68e2670 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:40.498110 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f022b1-185e-40b3-aa99-b3c0a68e2670 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:40.498379 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f022b1-185e-40b3-aa99-b3c0a68e2670 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:40.498997 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f022b1-185e-40b3-aa99-b3c0a68e2670 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:40.500587 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f022b1-185e-40b3-aa99-b3c0a68e2670 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:40.502418 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 624/1264] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:32:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:32:40.503297 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a9a8fc-6fce-404c-8064-aa610045432e service neutron] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:40.512318 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a9a8fc-6fce-404c-8064-aa610045432e service neutron] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:40.516856 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec9db26-716e-4a30-984e-26c204e8dcea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:40.517446 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec9db26-716e-4a30-984e-26c204e8dcea admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:40.517704 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec9db26-716e-4a30-984e-26c204e8dcea admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:40.517970 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec9db26-716e-4a30-984e-26c204e8dcea admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:40.518593 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec9db26-716e-4a30-984e-26c204e8dcea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:40.521391 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec9db26-716e-4a30-984e-26c204e8dcea admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:40.522535 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 641/1265] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:32:40 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:32:40.526911 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f022b1-185e-40b3-aa99-b3c0a68e2670 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:40.536748 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 625/1266] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:32:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:32:40.538238 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f022b1-185e-40b3-aa99-b3c0a68e2670 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:40.540263 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec9db26-716e-4a30-984e-26c204e8dcea admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:40.544543 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e835b23b-ff97-45a2-bc87-5c8e6559e7ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:40.545155 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e835b23b-ff97-45a2-bc87-5c8e6559e7ed admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:40.545527 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e835b23b-ff97-45a2-bc87-5c8e6559e7ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:40.545800 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e835b23b-ff97-45a2-bc87-5c8e6559e7ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:40.546634 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e835b23b-ff97-45a2-bc87-5c8e6559e7ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:40.548964 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e835b23b-ff97-45a2-bc87-5c8e6559e7ed admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:40.551279 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec9db26-716e-4a30-984e-26c204e8dcea admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:40.559855 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bb51c0-feda-4315-b902-af70d1a9d347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:40.563025 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bb51c0-feda-4315-b902-af70d1a9d347 admin admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:40.563155 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 642/1267] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:32:40 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:32:40.563448 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bb51c0-feda-4315-b902-af70d1a9d347 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:40.563448 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bb51c0-feda-4315-b902-af70d1a9d347 admin admin] PATH_INFO: `/v3/projects/a4d5cd8851dc4216b4e3e035a32decad/users/70d59ee6f94848308665a32099cf5255/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:40.563620 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e835b23b-ff97-45a2-bc87-5c8e6559e7ed admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:40.572370 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e835b23b-ff97-45a2-bc87-5c8e6559e7ed admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:40.577850 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a309506-7314-48bd-8ba2-48c1ec25ef80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:40.578416 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a309506-7314-48bd-8ba2-48c1ec25ef80 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:40.578714 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a309506-7314-48bd-8ba2-48c1ec25ef80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:40.579022 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a309506-7314-48bd-8ba2-48c1ec25ef80 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:40.579130 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bb51c0-feda-4315-b902-af70d1a9d347 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4d5cd8851dc4216b4e3e035a32decad, user_id=70d59ee6f94848308665a32099cf5255, role_id=ca1a5695f8bd499aa0b8cedaeecd2cc6)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:40.579686 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a309506-7314-48bd-8ba2-48c1ec25ef80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:40.581468 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a309506-7314-48bd-8ba2-48c1ec25ef80 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:40.582468 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bb51c0-feda-4315-b902-af70d1a9d347 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:40.598217 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 643/1268] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:32:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:32:40.598217 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a309506-7314-48bd-8ba2-48c1ec25ef80 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:40.598543 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 626/1269] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:32:40 2022] PUT /identity/v3/projects/a4d5cd8851dc4216b4e3e035a32decad/users/70d59ee6f94848308665a32099cf5255/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:32:40.598543 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bb51c0-feda-4315-b902-af70d1a9d347 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:40.611687 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a309506-7314-48bd-8ba2-48c1ec25ef80 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:40.654105 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:32:40.671810 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bb51c0-feda-4315-b902-af70d1a9d347 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:40.677304 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4b7824-9917-4fed-af6b-23ce5c230aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:40.677464 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee17dea-9913-4d9b-bf44-534976dc0c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:40.677923 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4b7824-9917-4fed-af6b-23ce5c230aee admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:40.678127 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee17dea-9913-4d9b-bf44-534976dc0c58 admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:40.678240 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4b7824-9917-4fed-af6b-23ce5c230aee admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:40.678405 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee17dea-9913-4d9b-bf44-534976dc0c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:40.678522 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4b7824-9917-4fed-af6b-23ce5c230aee admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:40.678675 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee17dea-9913-4d9b-bf44-534976dc0c58 admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:40.679138 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4b7824-9917-4fed-af6b-23ce5c230aee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:40.679346 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee17dea-9913-4d9b-bf44-534976dc0c58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:40.680758 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4b7824-9917-4fed-af6b-23ce5c230aee admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:40.680923 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee17dea-9913-4d9b-bf44-534976dc0c58 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:40.691799 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 627/1270] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:32:40 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:32:40.692780 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 644/1271] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:32:40 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:32:40.693786 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4b7824-9917-4fed-af6b-23ce5c230aee admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:40.696153 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee17dea-9913-4d9b-bf44-534976dc0c58 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:40.704551 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4b7824-9917-4fed-af6b-23ce5c230aee admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:40.707355 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee17dea-9913-4d9b-bf44-534976dc0c58 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:40.709297 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e0b612-b5fa-4b54-bc52-9a43ba12b507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:40.709897 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e0b612-b5fa-4b54-bc52-9a43ba12b507 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:40.710173 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e0b612-b5fa-4b54-bc52-9a43ba12b507 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:40.710443 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e0b612-b5fa-4b54-bc52-9a43ba12b507 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:40.711103 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e0b612-b5fa-4b54-bc52-9a43ba12b507 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:40.711900 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d2cb3f-dcc3-4e64-ae49-d73239e884e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:40.712706 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e0b612-b5fa-4b54-bc52-9a43ba12b507 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:40.712838 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d2cb3f-dcc3-4e64-ae49-d73239e884e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:40.712991 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d2cb3f-dcc3-4e64-ae49-d73239e884e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:40.713267 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d2cb3f-dcc3-4e64-ae49-d73239e884e4 admin admin] PATH_INFO: `/v3/projects/7573cb9c503243ab90b1649591d550e6/users/db2e978bcb824a5e9d593b861c9e3bd7/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:40.723612 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 645/1272] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:32:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:32:40.724687 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e0b612-b5fa-4b54-bc52-9a43ba12b507 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:40.732782 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d2cb3f-dcc3-4e64-ae49-d73239e884e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7573cb9c503243ab90b1649591d550e6, user_id=db2e978bcb824a5e9d593b861c9e3bd7, role_id=ca1a5695f8bd499aa0b8cedaeecd2cc6)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:40.735483 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d2cb3f-dcc3-4e64-ae49-d73239e884e4 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:40.736003 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e0b612-b5fa-4b54-bc52-9a43ba12b507 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:40.742668 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1e30e4-8948-4e7a-a504-138f49d5700f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:40.743315 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1e30e4-8948-4e7a-a504-138f49d5700f admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:40.743923 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1e30e4-8948-4e7a-a504-138f49d5700f admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:40.744232 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1e30e4-8948-4e7a-a504-138f49d5700f admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:40.745025 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1e30e4-8948-4e7a-a504-138f49d5700f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:40.746722 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1e30e4-8948-4e7a-a504-138f49d5700f admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:40.751230 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 628/1273] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:32:40 2022] PUT /identity/v3/projects/7573cb9c503243ab90b1649591d550e6/users/db2e978bcb824a5e9d593b861c9e3bd7/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:32:40.758273 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d2cb3f-dcc3-4e64-ae49-d73239e884e4 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:40.759842 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 646/1274] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:32:40 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:32:40.760211 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e2e3d1-0651-4283-9965-bff6894c16b5 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:40.760494 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e2e3d1-0651-4283-9965-bff6894c16b5 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:40.760846 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e2e3d1-0651-4283-9965-bff6894c16b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:40.783524 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-94e2e3d1-0651-4283-9965-bff6894c16b5 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:32:40.784751 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-94e2e3d1-0651-4283-9965-bff6894c16b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:32:40.817866 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:32:40.840306 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d2cb3f-dcc3-4e64-ae49-d73239e884e4 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:40.848671 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a02935d-1a96-48d8-b8f1-a439f4443970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:40.849666 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a02935d-1a96-48d8-b8f1-a439f4443970 admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:40.850009 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a02935d-1a96-48d8-b8f1-a439f4443970 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:40.850324 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a02935d-1a96-48d8-b8f1-a439f4443970 admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:40.850975 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a02935d-1a96-48d8-b8f1-a439f4443970 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:40.851778 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 647/1275] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:32:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:32:40.852734 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e2e3d1-0651-4283-9965-bff6894c16b5 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:40.864733 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e2e3d1-0651-4283-9965-bff6894c16b5 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:40.869778 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c66a7e9-8dc4-4b2e-a906-729ab8550c5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:40.870687 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c66a7e9-8dc4-4b2e-a906-729ab8550c5d admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:40.870802 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c66a7e9-8dc4-4b2e-a906-729ab8550c5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:40.871056 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c66a7e9-8dc4-4b2e-a906-729ab8550c5d admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:40.871801 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c66a7e9-8dc4-4b2e-a906-729ab8550c5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:40.872055 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a02935d-1a96-48d8-b8f1-a439f4443970 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:40.873607 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c66a7e9-8dc4-4b2e-a906-729ab8550c5d admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:40.888201 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 648/1276] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:32:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:32:40.888201 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c66a7e9-8dc4-4b2e-a906-729ab8550c5d admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:40.896515 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 629/1277] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:32:40 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:32:40.897367 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a02935d-1a96-48d8-b8f1-a439f4443970 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:40.897526 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c66a7e9-8dc4-4b2e-a906-729ab8550c5d admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:40.902289 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0b3bd0-4478-4226-ba9b-1d6f9f116c16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:40.903031 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0b3bd0-4478-4226-ba9b-1d6f9f116c16 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:40.903428 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0b3bd0-4478-4226-ba9b-1d6f9f116c16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:40.903791 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0b3bd0-4478-4226-ba9b-1d6f9f116c16 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:40.904520 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0b3bd0-4478-4226-ba9b-1d6f9f116c16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:40.906246 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0b3bd0-4478-4226-ba9b-1d6f9f116c16 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:40.911615 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a02935d-1a96-48d8-b8f1-a439f4443970 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:40.916543 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e29dccb-fbeb-40a3-b138-c62031b9e782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:40.917212 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e29dccb-fbeb-40a3-b138-c62031b9e782 admin admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:40.917492 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e29dccb-fbeb-40a3-b138-c62031b9e782 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:40.917765 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e29dccb-fbeb-40a3-b138-c62031b9e782 admin admin] PATH_INFO: `/v3/projects/9235562eee904084bc2ca5db0aff85ca/users/4ef0091787ff4f57bc5c05bfa528ee81/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:40.921526 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 649/1278] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:32:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:32:40.921526 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14505c72-94d1-45e3-b3d7-e1e13638355e None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:40.921526 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14505c72-94d1-45e3-b3d7-e1e13638355e None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:40.921526 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14505c72-94d1-45e3-b3d7-e1e13638355e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:40.937685 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e29dccb-fbeb-40a3-b138-c62031b9e782 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9235562eee904084bc2ca5db0aff85ca, user_id=4ef0091787ff4f57bc5c05bfa528ee81, role_id=ca1a5695f8bd499aa0b8cedaeecd2cc6)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:40.940796 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e29dccb-fbeb-40a3-b138-c62031b9e782 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:40.948420 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-14505c72-94d1-45e3-b3d7-e1e13638355e None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:32:40.949626 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-14505c72-94d1-45e3-b3d7-e1e13638355e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:32:40.957810 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 650/1279] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:32:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:32:40.958519 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14505c72-94d1-45e3-b3d7-e1e13638355e None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:40.959252 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 630/1280] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:32:40 2022] PUT /identity/v3/projects/9235562eee904084bc2ca5db0aff85ca/users/4ef0091787ff4f57bc5c05bfa528ee81/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:32:40.960296 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e29dccb-fbeb-40a3-b138-c62031b9e782 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:40.980092 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14505c72-94d1-45e3-b3d7-e1e13638355e None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:41.037098 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e29dccb-fbeb-40a3-b138-c62031b9e782 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:41.040730 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec50bf5-3f24-4362-8a07-350e84a3d4b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:41.041342 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec50bf5-3f24-4362-8a07-350e84a3d4b5 service neutron] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:41.041615 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec50bf5-3f24-4362-8a07-350e84a3d4b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:41.041882 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec50bf5-3f24-4362-8a07-350e84a3d4b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:41.048266 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1548f81f-61e2-4510-8240-24909fd84981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:41.049132 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1548f81f-61e2-4510-8240-24909fd84981 admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:41.049558 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1548f81f-61e2-4510-8240-24909fd84981 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:41.049980 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1548f81f-61e2-4510-8240-24909fd84981 admin admin] PATH_INFO: `/v3/projects/b9a2bf9a5a484523a700e4e6c568d4de/users/d6c1beb321e44908ab987b17f7b74484/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:41.065686 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1548f81f-61e2-4510-8240-24909fd84981 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9a2bf9a5a484523a700e4e6c568d4de, user_id=d6c1beb321e44908ab987b17f7b74484, role_id=ca1a5695f8bd499aa0b8cedaeecd2cc6)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:41.069328 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1548f81f-61e2-4510-8240-24909fd84981 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:41.089076 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 651/1281] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:32:40 2022] PUT /identity/v3/projects/b9a2bf9a5a484523a700e4e6c568d4de/users/d6c1beb321e44908ab987b17f7b74484/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:32:41.089076 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1548f81f-61e2-4510-8240-24909fd84981 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:41.093452 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:32:41.113528 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec50bf5-3f24-4362-8a07-350e84a3d4b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:41.115750 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec50bf5-3f24-4362-8a07-350e84a3d4b5 service neutron] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:41.131224 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 631/1282] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:32:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:32:41.132628 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec50bf5-3f24-4362-8a07-350e84a3d4b5 service neutron] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:41.149733 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec50bf5-3f24-4362-8a07-350e84a3d4b5 service neutron] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:41.170033 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1548f81f-61e2-4510-8240-24909fd84981 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:41.177285 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93302ac7-c46b-4ad3-90ac-ad9041b86d30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:41.178221 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93302ac7-c46b-4ad3-90ac-ad9041b86d30 admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:41.178576 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93302ac7-c46b-4ad3-90ac-ad9041b86d30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:41.179067 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93302ac7-c46b-4ad3-90ac-ad9041b86d30 admin admin] PATH_INFO: `/v3/projects/f358b4885deb4fd2a377cc4f358ed2d4/users/fba837d05c3e441ab2500f342edd37d1/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:41.200110 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93302ac7-c46b-4ad3-90ac-ad9041b86d30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f358b4885deb4fd2a377cc4f358ed2d4, user_id=fba837d05c3e441ab2500f342edd37d1, role_id=ca1a5695f8bd499aa0b8cedaeecd2cc6)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:41.204517 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93302ac7-c46b-4ad3-90ac-ad9041b86d30 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:41.235013 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 652/1283] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:32:41 2022] PUT /identity/v3/projects/f358b4885deb4fd2a377cc4f358ed2d4/users/fba837d05c3e441ab2500f342edd37d1/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:32:41.235013 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93302ac7-c46b-4ad3-90ac-ad9041b86d30 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:41.235436 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f29f44-0ac8-4de6-904d-649ac67ee1f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:41.235436 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f29f44-0ac8-4de6-904d-649ac67ee1f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:41.235626 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f29f44-0ac8-4de6-904d-649ac67ee1f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:41.235918 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f29f44-0ac8-4de6-904d-649ac67ee1f5 admin admin] PATH_INFO: `/v3/projects/de0eb090bf44403983e0cbb99e038321/users/a6d1ace68aa84499a4929da60f3d71bf/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:41.250980 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:32:41.256687 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f29f44-0ac8-4de6-904d-649ac67ee1f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de0eb090bf44403983e0cbb99e038321, user_id=a6d1ace68aa84499a4929da60f3d71bf, role_id=ca1a5695f8bd499aa0b8cedaeecd2cc6)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:41.261294 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f29f44-0ac8-4de6-904d-649ac67ee1f5 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:41.284198 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 632/1284] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:32:41 2022] PUT /identity/v3/projects/de0eb090bf44403983e0cbb99e038321/users/a6d1ace68aa84499a4929da60f3d71bf/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:32:41.285223 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f29f44-0ac8-4de6-904d-649ac67ee1f5 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:41.340906 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93302ac7-c46b-4ad3-90ac-ad9041b86d30 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:41.365564 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f29f44-0ac8-4de6-904d-649ac67ee1f5 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:41.370345 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7688d978-f640-485c-bef5-522dde3e8918 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:41.371069 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7688d978-f640-485c-bef5-522dde3e8918 service neutron] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:41.371443 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7688d978-f640-485c-bef5-522dde3e8918 service neutron] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:41.371787 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7688d978-f640-485c-bef5-522dde3e8918 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:41.457811 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a974e60-91c2-497b-9603-9c19e7b86446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:41.459275 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a974e60-91c2-497b-9603-9c19e7b86446 admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:41.459518 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a974e60-91c2-497b-9603-9c19e7b86446 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:41.459882 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a974e60-91c2-497b-9603-9c19e7b86446 admin admin] PATH_INFO: `/v3/projects/f7c7ae7636c54e8d93df903ff10551df/users/729645f4a2c34c7d9a80b9e85773cbfd/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:41.486827 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a974e60-91c2-497b-9603-9c19e7b86446 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7c7ae7636c54e8d93df903ff10551df, user_id=729645f4a2c34c7d9a80b9e85773cbfd, role_id=ca1a5695f8bd499aa0b8cedaeecd2cc6)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:41.488338 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7688d978-f640-485c-bef5-522dde3e8918 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:41.489919 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7688d978-f640-485c-bef5-522dde3e8918 service neutron] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:41.490760 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a974e60-91c2-497b-9603-9c19e7b86446 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:41.504817 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 633/1285] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:32:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:32:41.506603 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1c9e4a-7dea-499c-8a27-0ab2c2ffd44e None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:41.507000 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1c9e4a-7dea-499c-8a27-0ab2c2ffd44e None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:41.507363 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1c9e4a-7dea-499c-8a27-0ab2c2ffd44e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:41.523009 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 653/1286] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:32:41 2022] PUT /identity/v3/projects/f7c7ae7636c54e8d93df903ff10551df/users/729645f4a2c34c7d9a80b9e85773cbfd/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:32:41.523009 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf0b722-c44b-419a-9d11-a0aa39091922 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:41.523009 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf0b722-c44b-419a-9d11-a0aa39091922 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:41.523009 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf0b722-c44b-419a-9d11-a0aa39091922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:41.540068 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-dd1c9e4a-7dea-499c-8a27-0ab2c2ffd44e None None] MFA Rules not processed for user `4ef0091787ff4f57bc5c05bfa528ee81`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:32:41.541681 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-dd1c9e4a-7dea-499c-8a27-0ab2c2ffd44e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:32:41.546529 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:32:41.551733 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-4cf0b722-c44b-419a-9d11-a0aa39091922 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:32:41.553369 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-4cf0b722-c44b-419a-9d11-a0aa39091922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:32:41.670238 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 634/1287] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:32:41 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:32:41.671681 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58daeb3b-4333-4858-8577-41915111baa6 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:41.671968 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58daeb3b-4333-4858-8577-41915111baa6 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:41.672243 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58daeb3b-4333-4858-8577-41915111baa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:41.675796 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 654/1288] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:32:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:32:41.677085 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab39740c-1a1e-489d-a713-c6ab2c30dd3e None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:41.677368 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab39740c-1a1e-489d-a713-c6ab2c30dd3e None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:41.677682 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab39740c-1a1e-489d-a713-c6ab2c30dd3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:41.701588 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-ab39740c-1a1e-489d-a713-c6ab2c30dd3e None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:32:41.703006 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-ab39740c-1a1e-489d-a713-c6ab2c30dd3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:32:41.710108 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-58daeb3b-4333-4858-8577-41915111baa6 None None] MFA Rules not processed for user `fba837d05c3e441ab2500f342edd37d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:32:41.711345 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 655/1289] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:32:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:32:41.712047 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-58daeb3b-4333-4858-8577-41915111baa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:32:41.716028 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc90b74-ca53-4759-98cd-99fb5493172d None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:41.716311 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc90b74-ca53-4759-98cd-99fb5493172d None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:41.716588 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc90b74-ca53-4759-98cd-99fb5493172d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:41.774341 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-cdc90b74-ca53-4759-98cd-99fb5493172d None None] MFA Rules not processed for user `729645f4a2c34c7d9a80b9e85773cbfd`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:32:41.776470 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-cdc90b74-ca53-4759-98cd-99fb5493172d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:32:41.807408 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:32:41.807962 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 635/1290] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:32:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:32:41.810540 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58daeb3b-4333-4858-8577-41915111baa6 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:41.828483 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58daeb3b-4333-4858-8577-41915111baa6 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:41.843931 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ec16a2-4dcd-4939-8f6f-b97d21e929ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:41.844555 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ec16a2-4dcd-4939-8f6f-b97d21e929ef admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:41.844840 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ec16a2-4dcd-4939-8f6f-b97d21e929ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:41.845404 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ec16a2-4dcd-4939-8f6f-b97d21e929ef admin admin] PATH_INFO: `/v3/projects` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:41.845753 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ec16a2-4dcd-4939-8f6f-b97d21e929ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:41.848492 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ec16a2-4dcd-4939-8f6f-b97d21e929ef admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:41.905334 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 656/1291] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:32:41 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:32:41.906670 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc90b74-ca53-4759-98cd-99fb5493172d None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:41.907791 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-cdc90b74-ca53-4759-98cd-99fb5493172d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:32:41.915699 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 636/1292] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:32:41 2022] POST /identity/v3/projects => generated 371 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:32:41.917118 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ec16a2-4dcd-4939-8f6f-b97d21e929ef admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:41.980567 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc90b74-ca53-4759-98cd-99fb5493172d None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:42.010519 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ec16a2-4dcd-4939-8f6f-b97d21e929ef admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:42.015640 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b29d2cd-f5cd-4fe5-bef7-5073cf5f03f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:42.016726 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b29d2cd-f5cd-4fe5-bef7-5073cf5f03f6 service neutron] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:42.017017 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b29d2cd-f5cd-4fe5-bef7-5073cf5f03f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:42.017330 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b29d2cd-f5cd-4fe5-bef7-5073cf5f03f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:42.045799 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd09b9bb-2f6e-44f0-9c53-cc4e3da9f546 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:42.046892 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd09b9bb-2f6e-44f0-9c53-cc4e3da9f546 service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:42.047274 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd09b9bb-2f6e-44f0-9c53-cc4e3da9f546 service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:42.047754 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd09b9bb-2f6e-44f0-9c53-cc4e3da9f546 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:42.066000 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:32:42.123356 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b29d2cd-f5cd-4fe5-bef7-5073cf5f03f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:42.125437 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b29d2cd-f5cd-4fe5-bef7-5073cf5f03f6 service neutron] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:42.134047 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd09b9bb-2f6e-44f0-9c53-cc4e3da9f546 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:42.136506 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd09b9bb-2f6e-44f0-9c53-cc4e3da9f546 service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:42.141394 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 637/1293] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:32:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:32:42.143343 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dc5726-8e56-4460-a888-9de68a4a6ffb None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:42.143696 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dc5726-8e56-4460-a888-9de68a4a6ffb None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:42.144043 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dc5726-8e56-4460-a888-9de68a4a6ffb None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:42.151837 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 638/1294] 10.222.0.15 () {54 vars in 961 bytes} [Mon Oct 31 12:32:42 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:32:42.152848 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dc5726-8e56-4460-a888-9de68a4a6ffb None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:42.167278 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 657/1295] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:32:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:32:42.169073 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd09b9bb-2f6e-44f0-9c53-cc4e3da9f546 service neutron] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:42.170396 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dc5726-8e56-4460-a888-9de68a4a6ffb None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:42.181634 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19372d7a-443d-442a-b016-fa88f76c5c78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:42.182383 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19372d7a-443d-442a-b016-fa88f76c5c78 admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:42.182551 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19372d7a-443d-442a-b016-fa88f76c5c78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:42.182768 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd09b9bb-2f6e-44f0-9c53-cc4e3da9f546 service neutron] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:42.183537 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19372d7a-443d-442a-b016-fa88f76c5c78 admin admin] PATH_INFO: `/v3/projects` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:42.184193 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19372d7a-443d-442a-b016-fa88f76c5c78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:42.186106 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19372d7a-443d-442a-b016-fa88f76c5c78 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:42.192922 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9858b9fe-a1fc-4603-99cb-67326d610d09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:42.194806 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9858b9fe-a1fc-4603-99cb-67326d610d09 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:42.195085 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9858b9fe-a1fc-4603-99cb-67326d610d09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:42.196001 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9858b9fe-a1fc-4603-99cb-67326d610d09 admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:42.198650 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9858b9fe-a1fc-4603-99cb-67326d610d09 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:42.211047 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9858b9fe-a1fc-4603-99cb-67326d610d09 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:42.239927 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 639/1296] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:32:42 2022] POST /identity/v3/projects => generated 363 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:32:42.242192 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d03e03e-7d95-4061-a8f2-1d8a03a5cb6f None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:42.242506 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d03e03e-7d95-4061-a8f2-1d8a03a5cb6f None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:42.242939 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d03e03e-7d95-4061-a8f2-1d8a03a5cb6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:42.265803 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 658/1297] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:32:42 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:32:42.267760 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9858b9fe-a1fc-4603-99cb-67326d610d09 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:42.294903 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9858b9fe-a1fc-4603-99cb-67326d610d09 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:42.314216 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-0d03e03e-7d95-4061-a8f2-1d8a03a5cb6f None None] MFA Rules not processed for user `99889b1dab0b471d9c77887cffefc652`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:32:42.319281 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:32:42.320491 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-0d03e03e-7d95-4061-a8f2-1d8a03a5cb6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:32:42.404147 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36120c92-e147-491a-ac7d-8e830d932071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:42.405294 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36120c92-e147-491a-ac7d-8e830d932071 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:42.405421 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36120c92-e147-491a-ac7d-8e830d932071 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:42.405860 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36120c92-e147-491a-ac7d-8e830d932071 admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:42.406848 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36120c92-e147-491a-ac7d-8e830d932071 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:42.409223 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36120c92-e147-491a-ac7d-8e830d932071 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:42.446505 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 659/1298] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:32:42 2022] POST /identity/v3/users => generated 288 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:32:42.449957 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36120c92-e147-491a-ac7d-8e830d932071 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:42.462125 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 640/1299] 10.222.0.15 () {58 vars in 1058 bytes} [Mon Oct 31 12:32:42 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:32:42.463607 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d03e03e-7d95-4061-a8f2-1d8a03a5cb6f None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:42.472247 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36120c92-e147-491a-ac7d-8e830d932071 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:42.476589 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d03e03e-7d95-4061-a8f2-1d8a03a5cb6f None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:42.487821 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e74cb88-91d5-4941-9bf1-3a1b2c1d351d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:42.488650 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e74cb88-91d5-4941-9bf1-3a1b2c1d351d admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:42.488807 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e74cb88-91d5-4941-9bf1-3a1b2c1d351d admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:42.489082 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e74cb88-91d5-4941-9bf1-3a1b2c1d351d admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:42.489727 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e74cb88-91d5-4941-9bf1-3a1b2c1d351d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:42.491203 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6852477-d2fc-4db5-8bf7-09f59e955969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:42.492026 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6852477-d2fc-4db5-8bf7-09f59e955969 admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:42.492936 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6852477-d2fc-4db5-8bf7-09f59e955969 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:42.493368 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6852477-d2fc-4db5-8bf7-09f59e955969 admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:42.494407 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6852477-d2fc-4db5-8bf7-09f59e955969 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:42.496299 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6852477-d2fc-4db5-8bf7-09f59e955969 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:42.496930 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e74cb88-91d5-4941-9bf1-3a1b2c1d351d admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:42.512411 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 641/1300] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:32:42 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:32:42.513567 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6852477-d2fc-4db5-8bf7-09f59e955969 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:42.528828 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 660/1301] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:32:42 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:32:42.530275 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e74cb88-91d5-4941-9bf1-3a1b2c1d351d admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:42.534250 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6852477-d2fc-4db5-8bf7-09f59e955969 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:42.547743 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4828a20-44b0-40a5-b53f-e4d5ff229f27 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99889b1dab0b471d9c77887cffefc652', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:42.550217 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4828a20-44b0-40a5-b53f-e4d5ff229f27 service nova] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:42.553022 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4828a20-44b0-40a5-b53f-e4d5ff229f27 service nova] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:42.555019 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4828a20-44b0-40a5-b53f-e4d5ff229f27 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:42.567326 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e74cb88-91d5-4941-9bf1-3a1b2c1d351d admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:42.579760 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca613fee-1262-4904-94da-51ce1175c0d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:42.584249 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca613fee-1262-4904-94da-51ce1175c0d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:42.584749 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca613fee-1262-4904-94da-51ce1175c0d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:42.584996 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca613fee-1262-4904-94da-51ce1175c0d7 admin admin] PATH_INFO: `/v3/projects/17fde387d0a94493b34bb5b20ee9aa4c/users/c358bd97a32f4ddb96dc0e71d4dc1209/roles/176c07b4fe3b4537bdfeb4be21b379c5` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:42.624239 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca613fee-1262-4904-94da-51ce1175c0d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17fde387d0a94493b34bb5b20ee9aa4c, user_id=c358bd97a32f4ddb96dc0e71d4dc1209, role_id=176c07b4fe3b4537bdfeb4be21b379c5)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:42.637788 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca613fee-1262-4904-94da-51ce1175c0d7 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:42.667453 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 661/1302] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:32:42 2022] PUT /identity/v3/projects/17fde387d0a94493b34bb5b20ee9aa4c/users/c358bd97a32f4ddb96dc0e71d4dc1209/roles/176c07b4fe3b4537bdfeb4be21b379c5 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:32:42.668867 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca613fee-1262-4904-94da-51ce1175c0d7 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:42.686800 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4828a20-44b0-40a5-b53f-e4d5ff229f27 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:42.689668 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4828a20-44b0-40a5-b53f-e4d5ff229f27 service nova] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:42.700785 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 642/1303] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:32:42 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:32:42.702204 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4828a20-44b0-40a5-b53f-e4d5ff229f27 service nova] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:42.714961 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4828a20-44b0-40a5-b53f-e4d5ff229f27 service nova] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:42.722695 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:32:42.836080 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca613fee-1262-4904-94da-51ce1175c0d7 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:42.852118 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f53c19-caa4-4aaa-a8f6-5fd0afb5291f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:42.852829 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f53c19-caa4-4aaa-a8f6-5fd0afb5291f admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:42.853865 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f53c19-caa4-4aaa-a8f6-5fd0afb5291f admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:42.854335 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f53c19-caa4-4aaa-a8f6-5fd0afb5291f admin admin] PATH_INFO: `/v3/projects/6381bbd756444723ac34e0b49551764e/users/e2600c56ca4748cda4f44843d7eb6dd1/roles/176c07b4fe3b4537bdfeb4be21b379c5` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:42.876411 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8dc71b-66b4-47f7-bfd9-4e3f9ca22a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:42.877549 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8dc71b-66b4-47f7-bfd9-4e3f9ca22a23 admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:42.877549 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8dc71b-66b4-47f7-bfd9-4e3f9ca22a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:42.880811 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8dc71b-66b4-47f7-bfd9-4e3f9ca22a23 admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:42.884915 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8dc71b-66b4-47f7-bfd9-4e3f9ca22a23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:42.887848 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8dc71b-66b4-47f7-bfd9-4e3f9ca22a23 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:42.892883 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f53c19-caa4-4aaa-a8f6-5fd0afb5291f admin admin] RBAC: Authorizing `identity:create_grant(project_id=6381bbd756444723ac34e0b49551764e, user_id=e2600c56ca4748cda4f44843d7eb6dd1, role_id=176c07b4fe3b4537bdfeb4be21b379c5)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:42.902534 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f53c19-caa4-4aaa-a8f6-5fd0afb5291f admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:42.939576 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 662/1304] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:32:42 2022] PUT /identity/v3/projects/6381bbd756444723ac34e0b49551764e/users/e2600c56ca4748cda4f44843d7eb6dd1/roles/176c07b4fe3b4537bdfeb4be21b379c5 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:32:42.941829 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f53c19-caa4-4aaa-a8f6-5fd0afb5291f admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:42.944808 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 643/1305] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:32:42 2022] GET /identity/v3/roles => generated 1420 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:32:42.950306 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8dc71b-66b4-47f7-bfd9-4e3f9ca22a23 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:42.967873 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8dc71b-66b4-47f7-bfd9-4e3f9ca22a23 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:43.075059 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:32:43.105535 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f53c19-caa4-4aaa-a8f6-5fd0afb5291f admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:43.119597 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4a84e7-6a27-49b4-8139-480e07777a35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:43.121170 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4a84e7-6a27-49b4-8139-480e07777a35 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:43.121451 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4a84e7-6a27-49b4-8139-480e07777a35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:43.172917 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4a84e7-6a27-49b4-8139-480e07777a35 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:43.172917 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4a84e7-6a27-49b4-8139-480e07777a35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:43.172917 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4a84e7-6a27-49b4-8139-480e07777a35 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:43.174481 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c31cd94-a1f4-4679-9c18-3a893b581d52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:43.174481 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c31cd94-a1f4-4679-9c18-3a893b581d52 admin admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:43.175140 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 663/1306] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:32:42 2022] GET /identity/v3/roles => generated 1420 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 31 12:32:43.175463 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c31cd94-a1f4-4679-9c18-3a893b581d52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:43.175463 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c31cd94-a1f4-4679-9c18-3a893b581d52 admin admin] PATH_INFO: `/v3/projects/17fde387d0a94493b34bb5b20ee9aa4c/users/c358bd97a32f4ddb96dc0e71d4dc1209/roles/176c07b4fe3b4537bdfeb4be21b379c5` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:43.176147 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4a84e7-6a27-49b4-8139-480e07777a35 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:43.180290 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c31cd94-a1f4-4679-9c18-3a893b581d52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17fde387d0a94493b34bb5b20ee9aa4c, user_id=c358bd97a32f4ddb96dc0e71d4dc1209, role_id=176c07b4fe3b4537bdfeb4be21b379c5)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:43.184278 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c31cd94-a1f4-4679-9c18-3a893b581d52 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:43.197399 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4a84e7-6a27-49b4-8139-480e07777a35 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:43.198677 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 644/1307] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:32:42 2022] PUT /identity/v3/projects/17fde387d0a94493b34bb5b20ee9aa4c/users/c358bd97a32f4ddb96dc0e71d4dc1209/roles/176c07b4fe3b4537bdfeb4be21b379c5 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:32:43.208988 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ada5bf-5d1c-4f6a-8fb0-0102860d8d24 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:43.209422 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ada5bf-5d1c-4f6a-8fb0-0102860d8d24 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:43.209795 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ada5bf-5d1c-4f6a-8fb0-0102860d8d24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:43.292877 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-17ada5bf-5d1c-4f6a-8fb0-0102860d8d24 None None] MFA Rules not processed for user `c358bd97a32f4ddb96dc0e71d4dc1209`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:32:43.294328 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-17ada5bf-5d1c-4f6a-8fb0-0102860d8d24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:32:43.348597 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f4710f-f602-4ca4-a1a0-12dbc86ae658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:43.349286 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f4710f-f602-4ca4-a1a0-12dbc86ae658 admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:43.349564 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f4710f-f602-4ca4-a1a0-12dbc86ae658 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:43.349838 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f4710f-f602-4ca4-a1a0-12dbc86ae658 admin admin] PATH_INFO: `/v3/projects/6381bbd756444723ac34e0b49551764e/users/e2600c56ca4748cda4f44843d7eb6dd1/roles/176c07b4fe3b4537bdfeb4be21b379c5` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:43.357449 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f4710f-f602-4ca4-a1a0-12dbc86ae658 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6381bbd756444723ac34e0b49551764e, user_id=e2600c56ca4748cda4f44843d7eb6dd1, role_id=176c07b4fe3b4537bdfeb4be21b379c5)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:43.366761 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f4710f-f602-4ca4-a1a0-12dbc86ae658 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:43.421705 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 664/1308] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:32:43 2022] PUT /identity/v3/projects/6381bbd756444723ac34e0b49551764e/users/e2600c56ca4748cda4f44843d7eb6dd1/roles/176c07b4fe3b4537bdfeb4be21b379c5 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:32:43.421705 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850552ae-5707-4665-a0d6-60740897d800 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:43.421705 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850552ae-5707-4665-a0d6-60740897d800 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:43.421705 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850552ae-5707-4665-a0d6-60740897d800 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:43.421705 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 665/1309] 10.222.0.15 () {54 vars in 912 bytes} [Mon Oct 31 12:32:43 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:32:43.425271 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1c172d-a596-4d9c-88bc-bb6f952a2c69 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:43.426124 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1c172d-a596-4d9c-88bc-bb6f952a2c69 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:43.426376 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1c172d-a596-4d9c-88bc-bb6f952a2c69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:43.465591 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 645/1310] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:32:43 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:32:43.466656 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6da55db-1f03-47b0-828b-a5086c371ff1 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:43.466950 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6da55db-1f03-47b0-828b-a5086c371ff1 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:43.467236 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6da55db-1f03-47b0-828b-a5086c371ff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:43.507933 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-6d1c172d-a596-4d9c-88bc-bb6f952a2c69 None None] MFA Rules not processed for user `e2600c56ca4748cda4f44843d7eb6dd1`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:32:43.509371 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-b6da55db-1f03-47b0-828b-a5086c371ff1 None None] MFA Rules not processed for user `99889b1dab0b471d9c77887cffefc652`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:32:43.510608 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-b6da55db-1f03-47b0-828b-a5086c371ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:32:43.512771 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-6d1c172d-a596-4d9c-88bc-bb6f952a2c69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:32:43.549096 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:32:43.661916 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 666/1311] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:32:43 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:32:43.664024 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 646/1312] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Oct 31 12:32:43 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:32:43.664408 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1c172d-a596-4d9c-88bc-bb6f952a2c69 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:43.668519 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6da55db-1f03-47b0-828b-a5086c371ff1 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:43.873232 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1c172d-a596-4d9c-88bc-bb6f952a2c69 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:43.880069 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6da55db-1f03-47b0-828b-a5086c371ff1 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:43.896944 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7a8987-3465-4585-b6ba-749cf12bb33d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d6d7ea077d49e8a66c6c6762499702', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:43.897213 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9215db85-133f-4d82-bad0-6c9459f1fbd7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d6d7ea077d49e8a66c6c6762499702', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:43.899793 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9215db85-133f-4d82-bad0-6c9459f1fbd7 service cinder] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:43.900890 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9215db85-133f-4d82-bad0-6c9459f1fbd7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:43.901157 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7a8987-3465-4585-b6ba-749cf12bb33d service cinder] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:43.901860 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7a8987-3465-4585-b6ba-749cf12bb33d service cinder] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:43.902407 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9215db85-133f-4d82-bad0-6c9459f1fbd7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:43.902946 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7a8987-3465-4585-b6ba-749cf12bb33d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:43.943337 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7a8987-3465-4585-b6ba-749cf12bb33d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:43.947946 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:32:43.951623 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7a8987-3465-4585-b6ba-749cf12bb33d service cinder] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:44.032742 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9215db85-133f-4d82-bad0-6c9459f1fbd7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:44.041719 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 647/1313] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:32:43 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:32:44.042167 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9215db85-133f-4d82-bad0-6c9459f1fbd7 service cinder] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:44.056111 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 667/1314] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:32:43 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:32:44.134993 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7a8987-3465-4585-b6ba-749cf12bb33d service cinder] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:44.157930 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7a8987-3465-4585-b6ba-749cf12bb33d service cinder] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:44.173296 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cbd24e-a6b4-4aa4-b667-5f09682caac4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d6d7ea077d49e8a66c6c6762499702', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:44.174235 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cbd24e-a6b4-4aa4-b667-5f09682caac4 service cinder] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:44.174572 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cbd24e-a6b4-4aa4-b667-5f09682caac4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:44.175196 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cbd24e-a6b4-4aa4-b667-5f09682caac4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:44.199712 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cbd24e-a6b4-4aa4-b667-5f09682caac4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:44.202511 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cbd24e-a6b4-4aa4-b667-5f09682caac4 service cinder] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:44.231916 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 648/1315] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:32:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:32:44.308427 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9215db85-133f-4d82-bad0-6c9459f1fbd7 service cinder] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:44.325105 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9215db85-133f-4d82-bad0-6c9459f1fbd7 service cinder] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:44.330326 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5827b620-55eb-4c10-a4a7-92df4bff054d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d6d7ea077d49e8a66c6c6762499702', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:44.331275 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5827b620-55eb-4c10-a4a7-92df4bff054d service cinder] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:44.332452 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5827b620-55eb-4c10-a4a7-92df4bff054d service cinder] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:44.333015 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5827b620-55eb-4c10-a4a7-92df4bff054d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:44.487010 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5827b620-55eb-4c10-a4a7-92df4bff054d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:44.491339 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5827b620-55eb-4c10-a4a7-92df4bff054d service cinder] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:44.534535 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 668/1316] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:32:44 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:32:45.653994 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cbd24e-a6b4-4aa4-b667-5f09682caac4 service cinder] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:45.792819 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cbd24e-a6b4-4aa4-b667-5f09682caac4 service cinder] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:45.798107 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f92e24-d901-4027-8599-0bb0dab1969f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:45.798759 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f92e24-d901-4027-8599-0bb0dab1969f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:45.799037 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f92e24-d901-4027-8599-0bb0dab1969f admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:45.799346 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f92e24-d901-4027-8599-0bb0dab1969f admin admin] PATH_INFO: `/v3/users/4ef0091787ff4f57bc5c05bfa528ee81` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:45.804459 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f92e24-d901-4027-8599-0bb0dab1969f admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ef0091787ff4f57bc5c05bfa528ee81)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:45.806534 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f92e24-d901-4027-8599-0bb0dab1969f admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:45.879532 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32b9368-3c14-4079-9571-187033dee88d None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:45.880128 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32b9368-3c14-4079-9571-187033dee88d None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:45.880644 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32b9368-3c14-4079-9571-187033dee88d None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:45.882107 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 669/1317] 10.222.0.15 () {54 vars in 961 bytes} [Mon Oct 31 12:32:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:32:45.897021 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7425c9-16b0-42b0-8233-76ebd58d368f None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:45.897398 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7425c9-16b0-42b0-8233-76ebd58d368f None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:45.897742 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7425c9-16b0-42b0-8233-76ebd58d368f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:45.903604 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:32:45.907476 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-d0f92e24-d901-4027-8599-0bb0dab1969f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ef0091787ff4f57bc5c05bfa528ee81'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:32:45.928136 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-d0f92e24-d901-4027-8599-0bb0dab1969f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ef0091787ff4f57bc5c05bfa528ee81'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:32:45.929593 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-d0f92e24-d901-4027-8599-0bb0dab1969f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ef0091787ff4f57bc5c05bfa528ee81'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:32:45.935177 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-0f7425c9-16b0-42b0-8233-76ebd58d368f None None] MFA Rules not processed for user `99889b1dab0b471d9c77887cffefc652`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:32:45.936824 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-0f7425c9-16b0-42b0-8233-76ebd58d368f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:32:45.966477 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-d0f92e24-d901-4027-8599-0bb0dab1969f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ef0091787ff4f57bc5c05bfa528ee81'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:32:45.971903 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 670/1318] 10.222.0.15 () {58 vars in 1058 bytes} [Mon Oct 31 12:32:45 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:32:45.978708 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41099d1c-1664-4e5e-a9d0-55e07993f339 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:45.980609 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41099d1c-1664-4e5e-a9d0-55e07993f339 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:45.981766 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41099d1c-1664-4e5e-a9d0-55e07993f339 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:45.985035 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 671/1319] 10.222.0.15 () {54 vars in 912 bytes} [Mon Oct 31 12:32:45 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:32:45.985794 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-d0f92e24-d901-4027-8599-0bb0dab1969f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ef0091787ff4f57bc5c05bfa528ee81'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:32:45.991293 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bf72a9-82cd-42af-b104-93a58dbcbac0 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:45.991663 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bf72a9-82cd-42af-b104-93a58dbcbac0 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:45.992006 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bf72a9-82cd-42af-b104-93a58dbcbac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:46.023142 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 649/1320] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:32:45 2022] DELETE /identity/v3/users/4ef0091787ff4f57bc5c05bfa528ee81 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:32:46.026029 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93116a11-d72b-4e04-9090-e5716645c7e3 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:46.026343 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93116a11-d72b-4e04-9090-e5716645c7e3 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:46.026733 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93116a11-d72b-4e04-9090-e5716645c7e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:46.059967 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-e5bf72a9-82cd-42af-b104-93a58dbcbac0 None None] MFA Rules not processed for user `99889b1dab0b471d9c77887cffefc652`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:32:46.061479 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-e5bf72a9-82cd-42af-b104-93a58dbcbac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:32:46.095067 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-93116a11-d72b-4e04-9090-e5716645c7e3 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:32:46.096451 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-93116a11-d72b-4e04-9090-e5716645c7e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:32:46.248459 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 672/1321] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Oct 31 12:32:45 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:32:46.257085 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bf72a9-82cd-42af-b104-93a58dbcbac0 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:46.259946 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-e5bf72a9-82cd-42af-b104-93a58dbcbac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:32:46.281305 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 650/1322] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:32:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:32:46.345330 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93116a11-d72b-4e04-9090-e5716645c7e3 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:46.347685 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-93116a11-d72b-4e04-9090-e5716645c7e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:32:46.377921 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:32:46.410072 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bf72a9-82cd-42af-b104-93a58dbcbac0 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:46.420373 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fbeb28-c737-4d0c-a681-cc29b0527f08 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d6d7ea077d49e8a66c6c6762499702', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:46.421299 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fbeb28-c737-4d0c-a681-cc29b0527f08 service cinder] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:46.421730 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fbeb28-c737-4d0c-a681-cc29b0527f08 service cinder] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:46.422099 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fbeb28-c737-4d0c-a681-cc29b0527f08 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:46.450493 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fbeb28-c737-4d0c-a681-cc29b0527f08 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:46.454046 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fbeb28-c737-4d0c-a681-cc29b0527f08 service cinder] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:46.486714 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 673/1323] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:32:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 31 12:32:46.504891 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93116a11-d72b-4e04-9090-e5716645c7e3 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:46.514716 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a3f876-0cb4-404b-83e8-219d22395e5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:46.515377 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a3f876-0cb4-404b-83e8-219d22395e5e service neutron] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:46.515659 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a3f876-0cb4-404b-83e8-219d22395e5e service neutron] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:46.515931 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a3f876-0cb4-404b-83e8-219d22395e5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:46.539497 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a3f876-0cb4-404b-83e8-219d22395e5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:46.540950 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a3f876-0cb4-404b-83e8-219d22395e5e service neutron] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:46.560433 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 651/1324] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:32:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:32:48.215447 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fbeb28-c737-4d0c-a681-cc29b0527f08 service cinder] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:48.218175 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-45fbeb28-c737-4d0c-a681-cc29b0527f08 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:32:48.240155 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fbeb28-c737-4d0c-a681-cc29b0527f08 service cinder] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:48.251400 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaaecb8-ccb5-4143-adf5-2eb75e4c3a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:48.252896 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaaecb8-ccb5-4143-adf5-2eb75e4c3a2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:48.254515 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaaecb8-ccb5-4143-adf5-2eb75e4c3a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:48.254927 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaaecb8-ccb5-4143-adf5-2eb75e4c3a2a admin admin] PATH_INFO: `/v3/users/fba837d05c3e441ab2500f342edd37d1` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:48.259745 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaaecb8-ccb5-4143-adf5-2eb75e4c3a2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=fba837d05c3e441ab2500f342edd37d1)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:48.261076 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: WARNING py.warnings [None req-fcaaecb8-ccb5-4143-adf5-2eb75e4c3a2a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 31 12:32:48.261076 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: warnings.warn(msg) Oct 31 12:32:48.261076 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: Oct 31 12:32:48.264756 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaaecb8-ccb5-4143-adf5-2eb75e4c3a2a admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:48.415402 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-fcaaecb8-ccb5-4143-adf5-2eb75e4c3a2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fba837d05c3e441ab2500f342edd37d1'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:32:48.478906 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-fcaaecb8-ccb5-4143-adf5-2eb75e4c3a2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fba837d05c3e441ab2500f342edd37d1'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:32:48.480862 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-fcaaecb8-ccb5-4143-adf5-2eb75e4c3a2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fba837d05c3e441ab2500f342edd37d1'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:32:48.489154 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-fcaaecb8-ccb5-4143-adf5-2eb75e4c3a2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fba837d05c3e441ab2500f342edd37d1'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:32:48.538938 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-fcaaecb8-ccb5-4143-adf5-2eb75e4c3a2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fba837d05c3e441ab2500f342edd37d1'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:32:48.572928 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 674/1325] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:32:48 2022] DELETE /identity/v3/users/fba837d05c3e441ab2500f342edd37d1 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:32:48.575373 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca6f8db-f4b8-4f2a-abf1-8946f1bef483 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:48.575920 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca6f8db-f4b8-4f2a-abf1-8946f1bef483 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:48.576128 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca6f8db-f4b8-4f2a-abf1-8946f1bef483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:48.619122 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-eca6f8db-f4b8-4f2a-abf1-8946f1bef483 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:32:48.620943 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-eca6f8db-f4b8-4f2a-abf1-8946f1bef483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:32:48.818333 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 652/1326] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:32:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:32:48.837042 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaaecb8-ccb5-4143-adf5-2eb75e4c3a2a admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:48.839359 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-fcaaecb8-ccb5-4143-adf5-2eb75e4c3a2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:32:49.030140 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaaecb8-ccb5-4143-adf5-2eb75e4c3a2a admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:49.044871 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e604c38-d8d8-49a6-9d44-06ee3084a2fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:49.046082 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e604c38-d8d8-49a6-9d44-06ee3084a2fe service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:49.047583 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e604c38-d8d8-49a6-9d44-06ee3084a2fe service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:49.047907 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e604c38-d8d8-49a6-9d44-06ee3084a2fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:49.075389 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e604c38-d8d8-49a6-9d44-06ee3084a2fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:49.076890 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e604c38-d8d8-49a6-9d44-06ee3084a2fe service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:49.107281 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 675/1327] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:32:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 31 12:32:50.213408 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca6f8db-f4b8-4f2a-abf1-8946f1bef483 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:50.215272 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-eca6f8db-f4b8-4f2a-abf1-8946f1bef483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:32:50.239947 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca6f8db-f4b8-4f2a-abf1-8946f1bef483 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:50.261678 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee4ded6-e2bd-4fa4-a161-b0a6b2c65328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:50.262401 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee4ded6-e2bd-4fa4-a161-b0a6b2c65328 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:50.262679 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee4ded6-e2bd-4fa4-a161-b0a6b2c65328 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:50.262952 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee4ded6-e2bd-4fa4-a161-b0a6b2c65328 admin admin] PATH_INFO: `/v3/projects/9235562eee904084bc2ca5db0aff85ca` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:50.264157 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee4ded6-e2bd-4fa4-a161-b0a6b2c65328 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9235562eee904084bc2ca5db0aff85ca)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:50.265188 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: WARNING py.warnings [None req-8ee4ded6-e2bd-4fa4-a161-b0a6b2c65328 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 31 12:32:50.265188 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: warnings.warn(msg) Oct 31 12:32:50.265188 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: Oct 31 12:32:50.266452 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee4ded6-e2bd-4fa4-a161-b0a6b2c65328 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:50.268352 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:32:50.309915 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e604c38-d8d8-49a6-9d44-06ee3084a2fe service neutron] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:50.312391 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-5e604c38-d8d8-49a6-9d44-06ee3084a2fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:32:50.353860 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e604c38-d8d8-49a6-9d44-06ee3084a2fe service neutron] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:50.387188 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fcec36-f894-4102-a8d8-579ae0c3e0a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:50.387866 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fcec36-f894-4102-a8d8-579ae0c3e0a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:50.388155 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fcec36-f894-4102-a8d8-579ae0c3e0a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:50.388433 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fcec36-f894-4102-a8d8-579ae0c3e0a1 admin admin] PATH_INFO: `/v3/projects/f358b4885deb4fd2a377cc4f358ed2d4` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:50.406727 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fcec36-f894-4102-a8d8-579ae0c3e0a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f358b4885deb4fd2a377cc4f358ed2d4)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:50.409677 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fcec36-f894-4102-a8d8-579ae0c3e0a1 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:50.514687 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-8ee4ded6-e2bd-4fa4-a161-b0a6b2c65328 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9235562eee904084bc2ca5db0aff85ca', 'request_id': 'req-8ee4ded6-e2bd-4fa4-a161-b0a6b2c65328'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:32:50.539031 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-8ee4ded6-e2bd-4fa4-a161-b0a6b2c65328 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9235562eee904084bc2ca5db0aff85ca', 'request_id': 'req-8ee4ded6-e2bd-4fa4-a161-b0a6b2c65328'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:32:50.560862 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-8ee4ded6-e2bd-4fa4-a161-b0a6b2c65328 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9235562eee904084bc2ca5db0aff85ca', 'request_id': 'req-8ee4ded6-e2bd-4fa4-a161-b0a6b2c65328'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:32:50.623041 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-e2fcec36-f894-4102-a8d8-579ae0c3e0a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f358b4885deb4fd2a377cc4f358ed2d4', 'request_id': 'req-e2fcec36-f894-4102-a8d8-579ae0c3e0a1'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:32:50.631838 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:32:50.656476 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-e2fcec36-f894-4102-a8d8-579ae0c3e0a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f358b4885deb4fd2a377cc4f358ed2d4', 'request_id': 'req-e2fcec36-f894-4102-a8d8-579ae0c3e0a1'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:32:50.658371 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-8ee4ded6-e2bd-4fa4-a161-b0a6b2c65328 admin admin] The token cache is being invalidate because project 9235562eee904084bc2ca5db0aff85ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:32:50.660869 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-8ee4ded6-e2bd-4fa4-a161-b0a6b2c65328 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:32:50.662025 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-8ee4ded6-e2bd-4fa4-a161-b0a6b2c65328 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:32:50.669304 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 653/1328] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:32:50 2022] DELETE /identity/v3/projects/9235562eee904084bc2ca5db0aff85ca => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:32:50.673243 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee4ded6-e2bd-4fa4-a161-b0a6b2c65328 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:50.676294 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-8ee4ded6-e2bd-4fa4-a161-b0a6b2c65328 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:32:50.681596 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-e2fcec36-f894-4102-a8d8-579ae0c3e0a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f358b4885deb4fd2a377cc4f358ed2d4', 'request_id': 'req-e2fcec36-f894-4102-a8d8-579ae0c3e0a1'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:32:50.721078 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-e2fcec36-f894-4102-a8d8-579ae0c3e0a1 admin admin] The token cache is being invalidate because project f358b4885deb4fd2a377cc4f358ed2d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:32:50.725383 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-e2fcec36-f894-4102-a8d8-579ae0c3e0a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:32:50.730195 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-e2fcec36-f894-4102-a8d8-579ae0c3e0a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:32:50.735429 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 676/1329] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:32:50 2022] DELETE /identity/v3/projects/f358b4885deb4fd2a377cc4f358ed2d4 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 31 12:32:50.741591 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87c93a5-7e42-419a-9e6f-386535345e8d None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:50.743219 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87c93a5-7e42-419a-9e6f-386535345e8d None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:50.760757 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87c93a5-7e42-419a-9e6f-386535345e8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:50.843278 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-b87c93a5-7e42-419a-9e6f-386535345e8d None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:32:50.846058 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-b87c93a5-7e42-419a-9e6f-386535345e8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:32:50.938357 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee4ded6-e2bd-4fa4-a161-b0a6b2c65328 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:51.054641 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 677/1330] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:32:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:32:51.059086 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616f152f-a3d4-4584-b005-edc26a9d81f0 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:51.059574 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616f152f-a3d4-4584-b005-edc26a9d81f0 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:51.059899 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616f152f-a3d4-4584-b005-edc26a9d81f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:51.103387 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-616f152f-a3d4-4584-b005-edc26a9d81f0 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:32:51.104642 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-616f152f-a3d4-4584-b005-edc26a9d81f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:32:51.114730 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e579220f-e74a-4dd6-b6bc-a66bf8a7b979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:51.115950 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e579220f-e74a-4dd6-b6bc-a66bf8a7b979 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:51.116234 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e579220f-e74a-4dd6-b6bc-a66bf8a7b979 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:51.116514 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e579220f-e74a-4dd6-b6bc-a66bf8a7b979 admin admin] PATH_INFO: `/v3/users/e2600c56ca4748cda4f44843d7eb6dd1` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:51.119991 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e579220f-e74a-4dd6-b6bc-a66bf8a7b979 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2600c56ca4748cda4f44843d7eb6dd1)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:51.125121 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e579220f-e74a-4dd6-b6bc-a66bf8a7b979 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:51.128761 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 678/1331] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:32:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:32:51.136294 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616f152f-a3d4-4584-b005-edc26a9d81f0 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:51.158483 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616f152f-a3d4-4584-b005-edc26a9d81f0 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:51.179363 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a917df-aab3-4e1d-9515-dcd4647c03af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:51.182433 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a917df-aab3-4e1d-9515-dcd4647c03af admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:51.182772 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a917df-aab3-4e1d-9515-dcd4647c03af admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:51.183052 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a917df-aab3-4e1d-9515-dcd4647c03af admin admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:51.183788 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a917df-aab3-4e1d-9515-dcd4647c03af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:32:51.184099 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a917df-aab3-4e1d-9515-dcd4647c03af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:51.195595 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a917df-aab3-4e1d-9515-dcd4647c03af admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:51.222393 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 679/1332] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Oct 31 12:32:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:32:51.239510 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a917df-aab3-4e1d-9515-dcd4647c03af admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:51.241914 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:32:51.261440 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a917df-aab3-4e1d-9515-dcd4647c03af admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:51.261976 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-e579220f-e74a-4dd6-b6bc-a66bf8a7b979 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2600c56ca4748cda4f44843d7eb6dd1'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:32:51.272840 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb2d335-0d94-414a-9b43-6dafd6d2a4cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:51.275646 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb2d335-0d94-414a-9b43-6dafd6d2a4cf admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:51.276112 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb2d335-0d94-414a-9b43-6dafd6d2a4cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:51.276479 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb2d335-0d94-414a-9b43-6dafd6d2a4cf admin admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:51.278675 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb2d335-0d94-414a-9b43-6dafd6d2a4cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:51.282499 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-e579220f-e74a-4dd6-b6bc-a66bf8a7b979 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2600c56ca4748cda4f44843d7eb6dd1'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:32:51.283380 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-e579220f-e74a-4dd6-b6bc-a66bf8a7b979 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2600c56ca4748cda4f44843d7eb6dd1'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:32:51.283503 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb2d335-0d94-414a-9b43-6dafd6d2a4cf admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:51.297975 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-e579220f-e74a-4dd6-b6bc-a66bf8a7b979 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2600c56ca4748cda4f44843d7eb6dd1'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:32:51.305832 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-e579220f-e74a-4dd6-b6bc-a66bf8a7b979 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2600c56ca4748cda4f44843d7eb6dd1'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:32:51.314445 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 654/1333] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:32:50 2022] DELETE /identity/v3/users/e2600c56ca4748cda4f44843d7eb6dd1 => generated 0 bytes in 642 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:32:51.375831 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 680/1334] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:32:51 2022] POST /identity/v3/projects => generated 365 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:32:51.382131 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e579220f-e74a-4dd6-b6bc-a66bf8a7b979 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:51.385388 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-e579220f-e74a-4dd6-b6bc-a66bf8a7b979 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:32:51.532275 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e579220f-e74a-4dd6-b6bc-a66bf8a7b979 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:51.615841 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beffd6a3-d9d2-44a6-902f-cf37e0ec9a68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:51.616487 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beffd6a3-d9d2-44a6-902f-cf37e0ec9a68 admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:51.616833 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beffd6a3-d9d2-44a6-902f-cf37e0ec9a68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:51.617208 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beffd6a3-d9d2-44a6-902f-cf37e0ec9a68 admin admin] PATH_INFO: `/v3/users` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:51.618179 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beffd6a3-d9d2-44a6-902f-cf37e0ec9a68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:51.622828 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beffd6a3-d9d2-44a6-902f-cf37e0ec9a68 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:51.677198 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 655/1335] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:32:51 2022] POST /identity/v3/users => generated 283 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 31 12:32:51.678539 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb2d335-0d94-414a-9b43-6dafd6d2a4cf admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:51.690126 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb2d335-0d94-414a-9b43-6dafd6d2a4cf admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:51.705709 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13c6bdb-683d-4c25-80fb-d86f718ccd71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:51.706926 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13c6bdb-683d-4c25-80fb-d86f718ccd71 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:51.707250 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13c6bdb-683d-4c25-80fb-d86f718ccd71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:51.707523 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13c6bdb-683d-4c25-80fb-d86f718ccd71 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:51.708173 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13c6bdb-683d-4c25-80fb-d86f718ccd71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:51.713974 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13c6bdb-683d-4c25-80fb-d86f718ccd71 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:51.737126 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 681/1336] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:32:51 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:32:51.739867 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beffd6a3-d9d2-44a6-902f-cf37e0ec9a68 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:51.752858 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beffd6a3-d9d2-44a6-902f-cf37e0ec9a68 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:51.776893 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49acff5e-9758-4263-88af-22e865832715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:51.777557 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49acff5e-9758-4263-88af-22e865832715 admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:51.777834 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49acff5e-9758-4263-88af-22e865832715 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:51.778099 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49acff5e-9758-4263-88af-22e865832715 admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:51.778745 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49acff5e-9758-4263-88af-22e865832715 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:51.780448 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49acff5e-9758-4263-88af-22e865832715 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:51.809063 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 656/1337] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:32:51 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:32:51.813244 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13c6bdb-683d-4c25-80fb-d86f718ccd71 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:51.825543 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13c6bdb-683d-4c25-80fb-d86f718ccd71 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:51.836842 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c8b97e-1f56-4a13-881e-3847b810f2fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:51.837776 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c8b97e-1f56-4a13-881e-3847b810f2fa admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:51.838135 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c8b97e-1f56-4a13-881e-3847b810f2fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:51.838453 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c8b97e-1f56-4a13-881e-3847b810f2fa admin admin] PATH_INFO: `/v3/projects/9922c71d9dc44483b6febddf439bdc84/users/ff088f9fdce8440a8ce6cda7d2094b26/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:51.883067 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c8b97e-1f56-4a13-881e-3847b810f2fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=9922c71d9dc44483b6febddf439bdc84, user_id=ff088f9fdce8440a8ce6cda7d2094b26, role_id=ca1a5695f8bd499aa0b8cedaeecd2cc6)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:51.896287 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c8b97e-1f56-4a13-881e-3847b810f2fa admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:51.945220 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 682/1338] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:32:51 2022] PUT /identity/v3/projects/9922c71d9dc44483b6febddf439bdc84/users/ff088f9fdce8440a8ce6cda7d2094b26/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:32:51.957146 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5cc286-abcc-46f1-af5f-05f1260344d3 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:51.957501 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5cc286-abcc-46f1-af5f-05f1260344d3 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:51.957857 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5cc286-abcc-46f1-af5f-05f1260344d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:52.043509 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-ac5cc286-abcc-46f1-af5f-05f1260344d3 None None] MFA Rules not processed for user `ff088f9fdce8440a8ce6cda7d2094b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:32:52.045062 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-ac5cc286-abcc-46f1-af5f-05f1260344d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:32:52.130339 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:32:52.210572 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 657/1339] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:32:51 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:32:52.215319 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c8b97e-1f56-4a13-881e-3847b810f2fa admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:52.380435 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c8b97e-1f56-4a13-881e-3847b810f2fa admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:52.399478 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b65486-e2ac-40a5-9d8d-bef2f071d44e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:52.400733 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b65486-e2ac-40a5-9d8d-bef2f071d44e admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:52.401853 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b65486-e2ac-40a5-9d8d-bef2f071d44e admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:52.402251 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b65486-e2ac-40a5-9d8d-bef2f071d44e admin admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:52.403006 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b65486-e2ac-40a5-9d8d-bef2f071d44e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:52.405049 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b65486-e2ac-40a5-9d8d-bef2f071d44e admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:52.434327 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:32:52.472681 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 683/1340] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:32:52 2022] POST /identity/v3/projects => generated 363 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:32:52.476572 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5cc286-abcc-46f1-af5f-05f1260344d3 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:52.490188 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5cc286-abcc-46f1-af5f-05f1260344d3 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:52.613424 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c60eab8-f277-4457-8453-48b7504ef0b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:52.614220 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c60eab8-f277-4457-8453-48b7504ef0b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:52.614505 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c60eab8-f277-4457-8453-48b7504ef0b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:52.614775 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c60eab8-f277-4457-8453-48b7504ef0b1 admin admin] PATH_INFO: `/v3/users` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:52.615440 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c60eab8-f277-4457-8453-48b7504ef0b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:52.617376 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c60eab8-f277-4457-8453-48b7504ef0b1 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:52.687962 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 658/1341] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:32:52 2022] POST /identity/v3/users => generated 288 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 31 12:32:52.691844 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b65486-e2ac-40a5-9d8d-bef2f071d44e admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:52.709856 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b65486-e2ac-40a5-9d8d-bef2f071d44e admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:52.718863 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050dad26-7247-4cbc-8c74-1c02b188453b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:52.719688 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050dad26-7247-4cbc-8c74-1c02b188453b admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:52.719976 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050dad26-7247-4cbc-8c74-1c02b188453b admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:52.720243 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050dad26-7247-4cbc-8c74-1c02b188453b admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:52.721183 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050dad26-7247-4cbc-8c74-1c02b188453b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:52.723056 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050dad26-7247-4cbc-8c74-1c02b188453b admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:52.748319 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 684/1342] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:32:52 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:32:52.752334 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c60eab8-f277-4457-8453-48b7504ef0b1 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:52.767471 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c60eab8-f277-4457-8453-48b7504ef0b1 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:52.784224 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928b270a-e8bf-4d8d-9a2e-f2dbf13703a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:52.785468 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928b270a-e8bf-4d8d-9a2e-f2dbf13703a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:52.785799 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928b270a-e8bf-4d8d-9a2e-f2dbf13703a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:52.786072 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928b270a-e8bf-4d8d-9a2e-f2dbf13703a0 admin admin] PATH_INFO: `/v3/projects/8507a3a7b0fb4a21b3c528369a7ab8f9/users/5fcbe8cf40d747719937c4d264d4733e/roles/176c07b4fe3b4537bdfeb4be21b379c5` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:52.823492 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928b270a-e8bf-4d8d-9a2e-f2dbf13703a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8507a3a7b0fb4a21b3c528369a7ab8f9, user_id=5fcbe8cf40d747719937c4d264d4733e, role_id=176c07b4fe3b4537bdfeb4be21b379c5)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:52.826273 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928b270a-e8bf-4d8d-9a2e-f2dbf13703a0 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:52.859625 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 659/1343] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:32:52 2022] PUT /identity/v3/projects/8507a3a7b0fb4a21b3c528369a7ab8f9/users/5fcbe8cf40d747719937c4d264d4733e/roles/176c07b4fe3b4537bdfeb4be21b379c5 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:32:52.864336 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050dad26-7247-4cbc-8c74-1c02b188453b admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:52.875458 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050dad26-7247-4cbc-8c74-1c02b188453b admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:52.981672 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5375a684-93b8-4c83-ba3c-35dd98aac8a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:52.982323 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5375a684-93b8-4c83-ba3c-35dd98aac8a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:52.982603 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5375a684-93b8-4c83-ba3c-35dd98aac8a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:52.982877 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5375a684-93b8-4c83-ba3c-35dd98aac8a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:52.983550 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5375a684-93b8-4c83-ba3c-35dd98aac8a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:52.985143 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5375a684-93b8-4c83-ba3c-35dd98aac8a8 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:53.021898 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 685/1344] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:32:52 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 31 12:32:53.036211 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928b270a-e8bf-4d8d-9a2e-f2dbf13703a0 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:53.053284 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928b270a-e8bf-4d8d-9a2e-f2dbf13703a0 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:53.067671 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6148ff-b312-4482-b510-466b36e30802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:53.068302 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6148ff-b312-4482-b510-466b36e30802 admin admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:53.068581 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6148ff-b312-4482-b510-466b36e30802 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:53.068853 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6148ff-b312-4482-b510-466b36e30802 admin admin] PATH_INFO: `/v3/projects/8507a3a7b0fb4a21b3c528369a7ab8f9/users/5fcbe8cf40d747719937c4d264d4733e/roles/176c07b4fe3b4537bdfeb4be21b379c5` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:53.075814 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6148ff-b312-4482-b510-466b36e30802 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8507a3a7b0fb4a21b3c528369a7ab8f9, user_id=5fcbe8cf40d747719937c4d264d4733e, role_id=176c07b4fe3b4537bdfeb4be21b379c5)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:53.084352 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6148ff-b312-4482-b510-466b36e30802 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:53.115938 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 660/1345] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:32:53 2022] PUT /identity/v3/projects/8507a3a7b0fb4a21b3c528369a7ab8f9/users/5fcbe8cf40d747719937c4d264d4733e/roles/176c07b4fe3b4537bdfeb4be21b379c5 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:32:53.127818 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4c09b9-ddf1-412e-9d3e-e6460c4651ec None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:53.128040 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4c09b9-ddf1-412e-9d3e-e6460c4651ec None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:53.128351 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4c09b9-ddf1-412e-9d3e-e6460c4651ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:53.196652 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-8c4c09b9-ddf1-412e-9d3e-e6460c4651ec None None] MFA Rules not processed for user `5fcbe8cf40d747719937c4d264d4733e`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:32:53.198232 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-8c4c09b9-ddf1-412e-9d3e-e6460c4651ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:32:53.388093 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 686/1346] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:32:53 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 31 12:32:53.403002 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6148ff-b312-4482-b510-466b36e30802 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:53.405727 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-5d6148ff-b312-4482-b510-466b36e30802 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:32:53.509581 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6148ff-b312-4482-b510-466b36e30802 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:53.514585 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc500bca-f90f-423e-823a-5017bfd78295 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d6d7ea077d49e8a66c6c6762499702', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:53.515308 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc500bca-f90f-423e-823a-5017bfd78295 service cinder] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:53.516172 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc500bca-f90f-423e-823a-5017bfd78295 service cinder] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:53.516596 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc500bca-f90f-423e-823a-5017bfd78295 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:53.540521 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc500bca-f90f-423e-823a-5017bfd78295 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:53.542665 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc500bca-f90f-423e-823a-5017bfd78295 service cinder] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:53.620217 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 661/1347] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:32:53 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:32:53.943336 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e49fba-d221-4402-a63b-5e1d3a704cbe None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:53.943676 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e49fba-d221-4402-a63b-5e1d3a704cbe None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:53.944587 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e49fba-d221-4402-a63b-5e1d3a704cbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:54.045435 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-98e49fba-d221-4402-a63b-5e1d3a704cbe None None] MFA Rules not processed for user `d6c1beb321e44908ab987b17f7b74484`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:32:54.046743 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-98e49fba-d221-4402-a63b-5e1d3a704cbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:32:54.243997 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:32:54.305908 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 687/1348] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:32:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 364 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:32:54.322042 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc500bca-f90f-423e-823a-5017bfd78295 service cinder] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:54.323975 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-cc500bca-f90f-423e-823a-5017bfd78295 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:32:54.418875 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:32:54.556497 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc500bca-f90f-423e-823a-5017bfd78295 service cinder] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:54.563462 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58650bad-41c9-426d-9b9d-63716cb7c78c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99889b1dab0b471d9c77887cffefc652', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:54.563462 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58650bad-41c9-426d-9b9d-63716cb7c78c service nova] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:54.563738 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58650bad-41c9-426d-9b9d-63716cb7c78c service nova] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:54.563971 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58650bad-41c9-426d-9b9d-63716cb7c78c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:54.579604 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58650bad-41c9-426d-9b9d-63716cb7c78c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:54.584899 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58650bad-41c9-426d-9b9d-63716cb7c78c service nova] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:54.612327 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 662/1349] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:32:54 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:32:54.673096 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e49fba-d221-4402-a63b-5e1d3a704cbe None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:54.839536 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58650bad-41c9-426d-9b9d-63716cb7c78c service nova] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:54.841179 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-58650bad-41c9-426d-9b9d-63716cb7c78c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:32:54.842079 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e49fba-d221-4402-a63b-5e1d3a704cbe None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:54.855937 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d52c578-fb19-4275-b2b1-8d0665a69e2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:54.856714 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d52c578-fb19-4275-b2b1-8d0665a69e2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:54.857085 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d52c578-fb19-4275-b2b1-8d0665a69e2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:54.857408 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d52c578-fb19-4275-b2b1-8d0665a69e2a admin admin] PATH_INFO: `/v3/users/ff088f9fdce8440a8ce6cda7d2094b26` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:54.859470 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d52c578-fb19-4275-b2b1-8d0665a69e2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff088f9fdce8440a8ce6cda7d2094b26)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:54.862269 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d52c578-fb19-4275-b2b1-8d0665a69e2a admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:54.864813 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58650bad-41c9-426d-9b9d-63716cb7c78c service nova] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:54.870578 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1a4c1a-b09c-4043-b758-3e982ac3665f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:54.871695 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1a4c1a-b09c-4043-b758-3e982ac3665f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:54.872217 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1a4c1a-b09c-4043-b758-3e982ac3665f admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:54.872665 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1a4c1a-b09c-4043-b758-3e982ac3665f admin admin] PATH_INFO: `/v3/projects/6381bbd756444723ac34e0b49551764e` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:54.879745 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1a4c1a-b09c-4043-b758-3e982ac3665f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6381bbd756444723ac34e0b49551764e)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:54.882092 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1a4c1a-b09c-4043-b758-3e982ac3665f admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:54.976332 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-9d52c578-fb19-4275-b2b1-8d0665a69e2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff088f9fdce8440a8ce6cda7d2094b26'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:32:54.996234 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-9d52c578-fb19-4275-b2b1-8d0665a69e2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff088f9fdce8440a8ce6cda7d2094b26'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:32:54.997024 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-9d52c578-fb19-4275-b2b1-8d0665a69e2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff088f9fdce8440a8ce6cda7d2094b26'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:32:54.997971 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-9d52c578-fb19-4275-b2b1-8d0665a69e2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff088f9fdce8440a8ce6cda7d2094b26'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:32:55.016651 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-cd1a4c1a-b09c-4043-b758-3e982ac3665f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6381bbd756444723ac34e0b49551764e', 'request_id': 'req-cd1a4c1a-b09c-4043-b758-3e982ac3665f'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:32:55.043124 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-9d52c578-fb19-4275-b2b1-8d0665a69e2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff088f9fdce8440a8ce6cda7d2094b26'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:32:55.059139 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-cd1a4c1a-b09c-4043-b758-3e982ac3665f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6381bbd756444723ac34e0b49551764e', 'request_id': 'req-cd1a4c1a-b09c-4043-b758-3e982ac3665f'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:32:55.087571 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-cd1a4c1a-b09c-4043-b758-3e982ac3665f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6381bbd756444723ac34e0b49551764e', 'request_id': 'req-cd1a4c1a-b09c-4043-b758-3e982ac3665f'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:32:55.092835 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 688/1350] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:32:54 2022] DELETE /identity/v3/users/ff088f9fdce8440a8ce6cda7d2094b26 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:32:55.097956 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b580f6-9ef4-433e-9909-10c3fcf97ce5 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:55.098323 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b580f6-9ef4-433e-9909-10c3fcf97ce5 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:55.098600 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b580f6-9ef4-433e-9909-10c3fcf97ce5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:55.120811 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-cd1a4c1a-b09c-4043-b758-3e982ac3665f admin admin] The token cache is being invalidate because project 6381bbd756444723ac34e0b49551764e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:32:55.121106 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-cd1a4c1a-b09c-4043-b758-3e982ac3665f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:32:55.122317 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-cd1a4c1a-b09c-4043-b758-3e982ac3665f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:32:55.131118 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 663/1351] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:32:54 2022] DELETE /identity/v3/projects/6381bbd756444723ac34e0b49551764e => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:32:55.135040 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3306f079-8583-40c5-ab31-301b20eb04eb None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:55.135229 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3306f079-8583-40c5-ab31-301b20eb04eb None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:55.135505 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3306f079-8583-40c5-ab31-301b20eb04eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:55.144848 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-28b580f6-9ef4-433e-9909-10c3fcf97ce5 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:32:55.146875 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-28b580f6-9ef4-433e-9909-10c3fcf97ce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:32:55.177280 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-3306f079-8583-40c5-ab31-301b20eb04eb None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:32:55.178504 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-3306f079-8583-40c5-ab31-301b20eb04eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:32:55.281664 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 689/1352] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:32:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:32:55.334192 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 664/1353] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:32:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:32:55.342969 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b580f6-9ef4-433e-9909-10c3fcf97ce5 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:55.348710 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-28b580f6-9ef4-433e-9909-10c3fcf97ce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:32:55.356638 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a5ed6b-e65a-45f5-9760-cebd74956808 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:55.356990 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a5ed6b-e65a-45f5-9760-cebd74956808 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:55.357265 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a5ed6b-e65a-45f5-9760-cebd74956808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:55.371913 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:32:55.427751 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-a7a5ed6b-e65a-45f5-9760-cebd74956808 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:32:55.428984 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-a7a5ed6b-e65a-45f5-9760-cebd74956808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:32:55.461350 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 665/1354] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:32:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:32:55.462299 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a5ed6b-e65a-45f5-9760-cebd74956808 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:55.482893 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a5ed6b-e65a-45f5-9760-cebd74956808 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:55.495556 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3837ac-760c-4178-bdd5-5c4699c3b413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:55.496186 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3837ac-760c-4178-bdd5-5c4699c3b413 admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:55.496458 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3837ac-760c-4178-bdd5-5c4699c3b413 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:55.496723 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3837ac-760c-4178-bdd5-5c4699c3b413 admin admin] PATH_INFO: `/v3/domains` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:55.497376 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3837ac-760c-4178-bdd5-5c4699c3b413 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:32:55.497674 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3837ac-760c-4178-bdd5-5c4699c3b413 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:55.499302 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3837ac-760c-4178-bdd5-5c4699c3b413 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:55.506026 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b580f6-9ef4-433e-9909-10c3fcf97ce5 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:55.515874 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8a0453-179e-4818-98a4-6bfab14a963a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:55.516509 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8a0453-179e-4818-98a4-6bfab14a963a service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:55.516782 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8a0453-179e-4818-98a4-6bfab14a963a service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:55.517047 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8a0453-179e-4818-98a4-6bfab14a963a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:55.527726 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 666/1355] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Oct 31 12:32:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:32:55.532052 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3837ac-760c-4178-bdd5-5c4699c3b413 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:55.541488 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8a0453-179e-4818-98a4-6bfab14a963a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:55.542950 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8a0453-179e-4818-98a4-6bfab14a963a service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:55.552571 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3837ac-760c-4178-bdd5-5c4699c3b413 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:55.565276 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 690/1356] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:32:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:32:55.566008 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b753faea-9599-430c-8b6c-9873cb862abd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:55.566790 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b753faea-9599-430c-8b6c-9873cb862abd admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:55.566921 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b753faea-9599-430c-8b6c-9873cb862abd admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:55.567307 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b753faea-9599-430c-8b6c-9873cb862abd admin admin] PATH_INFO: `/v3/projects` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:55.567949 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b753faea-9599-430c-8b6c-9873cb862abd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:55.569847 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b753faea-9599-430c-8b6c-9873cb862abd admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:55.583351 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d4d029-8f15-475e-944d-9d8af44f608c None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:55.583635 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d4d029-8f15-475e-944d-9d8af44f608c None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:55.583907 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d4d029-8f15-475e-944d-9d8af44f608c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:55.623892 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-13d4d029-8f15-475e-944d-9d8af44f608c None None] MFA Rules not processed for user `b2c03f0365ee4a26b8c076ae365325b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:32:55.625085 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-13d4d029-8f15-475e-944d-9d8af44f608c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:32:55.629264 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:32:55.664803 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 667/1357] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:32:55 2022] POST /identity/v3/projects => generated 369 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:32:55.669144 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b753faea-9599-430c-8b6c-9873cb862abd admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:55.759821 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 691/1358] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:32:55 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:32:55.772430 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d4d029-8f15-475e-944d-9d8af44f608c None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:55.773875 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-13d4d029-8f15-475e-944d-9d8af44f608c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:32:55.858068 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b753faea-9599-430c-8b6c-9873cb862abd admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:55.871352 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec34ac09-a7a8-41e8-853b-3d28d105e8f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:55.872020 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec34ac09-a7a8-41e8-853b-3d28d105e8f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:55.872299 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec34ac09-a7a8-41e8-853b-3d28d105e8f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:55.872564 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec34ac09-a7a8-41e8-853b-3d28d105e8f1 admin admin] PATH_INFO: `/v3/users` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:55.873199 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec34ac09-a7a8-41e8-853b-3d28d105e8f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:55.882131 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec34ac09-a7a8-41e8-853b-3d28d105e8f1 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:55.944260 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 668/1359] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:32:55 2022] POST /identity/v3/users => generated 285 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:32:55.945248 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec34ac09-a7a8-41e8-853b-3d28d105e8f1 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:55.971424 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec34ac09-a7a8-41e8-853b-3d28d105e8f1 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:55.986718 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b88250-bf87-4e30-a1bb-83a46c1d557e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:55.989670 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b88250-bf87-4e30-a1bb-83a46c1d557e admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:55.989894 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b88250-bf87-4e30-a1bb-83a46c1d557e admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:55.991383 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b88250-bf87-4e30-a1bb-83a46c1d557e admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:55.992792 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b88250-bf87-4e30-a1bb-83a46c1d557e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:55.996891 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d4d029-8f15-475e-944d-9d8af44f608c None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:55.999338 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b88250-bf87-4e30-a1bb-83a46c1d557e admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:56.022784 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 669/1360] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:32:55 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:32:56.026847 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b88250-bf87-4e30-a1bb-83a46c1d557e admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:56.068162 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b88250-bf87-4e30-a1bb-83a46c1d557e admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:56.086995 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7826c166-bf5e-40db-a217-97ebcdb1c555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:56.091350 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7826c166-bf5e-40db-a217-97ebcdb1c555 admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:56.091778 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7826c166-bf5e-40db-a217-97ebcdb1c555 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:56.094497 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7826c166-bf5e-40db-a217-97ebcdb1c555 admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:56.096312 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7826c166-bf5e-40db-a217-97ebcdb1c555 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:56.098099 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7826c166-bf5e-40db-a217-97ebcdb1c555 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:56.143655 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 670/1361] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:32:56 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 31 12:32:56.144852 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:32:56.147198 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7826c166-bf5e-40db-a217-97ebcdb1c555 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:56.186679 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7826c166-bf5e-40db-a217-97ebcdb1c555 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:56.207315 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e047e395-cd9a-42b4-90df-00a5ca9db17a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:56.214159 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e047e395-cd9a-42b4-90df-00a5ca9db17a admin admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:56.215064 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e047e395-cd9a-42b4-90df-00a5ca9db17a admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:56.215368 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e047e395-cd9a-42b4-90df-00a5ca9db17a admin admin] PATH_INFO: `/v3/projects/cbf8f3193ba2448190df58e256e7e4f6/users/62bdb7d0f7484cc79c1c02fe1b79c671/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:56.241915 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2bd126-32af-48b2-9ea7-82eb92e28d6b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99889b1dab0b471d9c77887cffefc652', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:56.242545 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2bd126-32af-48b2-9ea7-82eb92e28d6b service nova] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:56.242819 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2bd126-32af-48b2-9ea7-82eb92e28d6b service nova] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:56.247784 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2bd126-32af-48b2-9ea7-82eb92e28d6b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:56.263566 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e047e395-cd9a-42b4-90df-00a5ca9db17a admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbf8f3193ba2448190df58e256e7e4f6, user_id=62bdb7d0f7484cc79c1c02fe1b79c671, role_id=ca1a5695f8bd499aa0b8cedaeecd2cc6)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:56.272159 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e047e395-cd9a-42b4-90df-00a5ca9db17a admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:56.314517 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 671/1362] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:32:56 2022] PUT /identity/v3/projects/cbf8f3193ba2448190df58e256e7e4f6/users/62bdb7d0f7484cc79c1c02fe1b79c671/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 31 12:32:56.316605 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b67f62-18bb-4c07-99c1-b549a881dbdf None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:56.317027 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b67f62-18bb-4c07-99c1-b549a881dbdf None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:56.317439 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b67f62-18bb-4c07-99c1-b549a881dbdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:56.381083 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-e4b67f62-18bb-4c07-99c1-b549a881dbdf None None] MFA Rules not processed for user `62bdb7d0f7484cc79c1c02fe1b79c671`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:32:56.385009 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-e4b67f62-18bb-4c07-99c1-b549a881dbdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:32:56.481345 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2bd126-32af-48b2-9ea7-82eb92e28d6b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:56.483326 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2bd126-32af-48b2-9ea7-82eb92e28d6b service nova] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:56.495223 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 692/1363] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:32:55 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 723 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 31 12:32:56.517966 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 672/1364] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:32:56 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:32:56.523342 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2bd126-32af-48b2-9ea7-82eb92e28d6b service nova] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:56.538768 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2bd126-32af-48b2-9ea7-82eb92e28d6b service nova] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:56.689644 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f8bba7-d8fa-4529-840b-f1606df1291a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:56.691117 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f8bba7-d8fa-4529-840b-f1606df1291a admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:56.691347 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f8bba7-d8fa-4529-840b-f1606df1291a admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:56.691669 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f8bba7-d8fa-4529-840b-f1606df1291a admin admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:56.693480 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f8bba7-d8fa-4529-840b-f1606df1291a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:56.697674 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f8bba7-d8fa-4529-840b-f1606df1291a admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:56.772834 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 693/1365] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:32:56 2022] POST /identity/v3/projects => generated 365 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:32:56.776659 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b67f62-18bb-4c07-99c1-b549a881dbdf None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:56.793139 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b67f62-18bb-4c07-99c1-b549a881dbdf None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:56.838176 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:32:56.918816 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f8bba7-d8fa-4529-840b-f1606df1291a admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:56.920831 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-f6f8bba7-d8fa-4529-840b-f1606df1291a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:32:56.934303 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe6b56b-3224-4b96-98c5-82aa977b3a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:56.935097 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe6b56b-3224-4b96-98c5-82aa977b3a27 admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:56.935282 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe6b56b-3224-4b96-98c5-82aa977b3a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:56.935564 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe6b56b-3224-4b96-98c5-82aa977b3a27 admin admin] PATH_INFO: `/v3/users` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:56.936104 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe6b56b-3224-4b96-98c5-82aa977b3a27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:56.938018 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe6b56b-3224-4b96-98c5-82aa977b3a27 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:56.945692 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f8bba7-d8fa-4529-840b-f1606df1291a admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:56.959136 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ebebaf-be0e-4bef-9a8f-5b0f92efc696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:56.960758 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ebebaf-be0e-4bef-9a8f-5b0f92efc696 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:56.962301 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ebebaf-be0e-4bef-9a8f-5b0f92efc696 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:56.962690 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ebebaf-be0e-4bef-9a8f-5b0f92efc696 admin admin] PATH_INFO: `/v3/projects/9922c71d9dc44483b6febddf439bdc84` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:56.966017 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ebebaf-be0e-4bef-9a8f-5b0f92efc696 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9922c71d9dc44483b6febddf439bdc84)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:56.970620 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ebebaf-be0e-4bef-9a8f-5b0f92efc696 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:56.986306 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 673/1366] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:32:56 2022] POST /identity/v3/users => generated 289 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:32:56.990498 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe6b56b-3224-4b96-98c5-82aa977b3a27 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:57.007543 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe6b56b-3224-4b96-98c5-82aa977b3a27 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:57.018650 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4370bb1-1750-4dc5-aa60-473e6f0b76e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:57.019821 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4370bb1-1750-4dc5-aa60-473e6f0b76e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:57.020099 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4370bb1-1750-4dc5-aa60-473e6f0b76e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:57.020386 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4370bb1-1750-4dc5-aa60-473e6f0b76e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:57.021274 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4370bb1-1750-4dc5-aa60-473e6f0b76e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:57.023469 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4370bb1-1750-4dc5-aa60-473e6f0b76e8 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:57.043084 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 674/1367] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:32:56 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 31 12:32:57.047123 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4370bb1-1750-4dc5-aa60-473e6f0b76e8 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:57.071378 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4370bb1-1750-4dc5-aa60-473e6f0b76e8 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:57.093795 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-c2ebebaf-be0e-4bef-9a8f-5b0f92efc696 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9922c71d9dc44483b6febddf439bdc84', 'request_id': 'req-c2ebebaf-be0e-4bef-9a8f-5b0f92efc696'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:32:57.110380 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-c2ebebaf-be0e-4bef-9a8f-5b0f92efc696 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9922c71d9dc44483b6febddf439bdc84', 'request_id': 'req-c2ebebaf-be0e-4bef-9a8f-5b0f92efc696'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:32:57.118322 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:32:57.130111 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-c2ebebaf-be0e-4bef-9a8f-5b0f92efc696 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9922c71d9dc44483b6febddf439bdc84', 'request_id': 'req-c2ebebaf-be0e-4bef-9a8f-5b0f92efc696'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:32:57.152099 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-c2ebebaf-be0e-4bef-9a8f-5b0f92efc696 admin admin] The token cache is being invalidate because project 9922c71d9dc44483b6febddf439bdc84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:32:57.155218 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-c2ebebaf-be0e-4bef-9a8f-5b0f92efc696 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:32:57.157955 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-c2ebebaf-be0e-4bef-9a8f-5b0f92efc696 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:32:57.166511 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 694/1368] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:32:56 2022] DELETE /identity/v3/projects/9922c71d9dc44483b6febddf439bdc84 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:32:57.168001 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ebebaf-be0e-4bef-9a8f-5b0f92efc696 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:57.169582 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-c2ebebaf-be0e-4bef-9a8f-5b0f92efc696 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:32:57.209534 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ffb55b-c7f2-4a10-a39f-7a56a22b6380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:57.210184 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ffb55b-c7f2-4a10-a39f-7a56a22b6380 admin admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:57.210457 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ffb55b-c7f2-4a10-a39f-7a56a22b6380 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:57.210725 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ffb55b-c7f2-4a10-a39f-7a56a22b6380 admin admin] PATH_INFO: `/v3/projects/80329fb711ad4e4ba5547af8f6f945e1/users/e4104a0759914252ba5d93394c38faf8/roles/176c07b4fe3b4537bdfeb4be21b379c5` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:57.243996 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ffb55b-c7f2-4a10-a39f-7a56a22b6380 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80329fb711ad4e4ba5547af8f6f945e1, user_id=e4104a0759914252ba5d93394c38faf8, role_id=176c07b4fe3b4537bdfeb4be21b379c5)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:57.246652 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ffb55b-c7f2-4a10-a39f-7a56a22b6380 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:57.287203 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 675/1369] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:32:57 2022] PUT /identity/v3/projects/80329fb711ad4e4ba5547af8f6f945e1/users/e4104a0759914252ba5d93394c38faf8/roles/176c07b4fe3b4537bdfeb4be21b379c5 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 31 12:32:57.291140 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ffb55b-c7f2-4a10-a39f-7a56a22b6380 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:57.294387 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-32ffb55b-c7f2-4a10-a39f-7a56a22b6380 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:32:57.339674 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ebebaf-be0e-4bef-9a8f-5b0f92efc696 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:57.451971 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ffb55b-c7f2-4a10-a39f-7a56a22b6380 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:57.468972 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3e6290-5e5e-4477-9162-1dbf129b7bc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:57.469892 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e6290-5e5e-4477-9162-1dbf129b7bc0 admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:57.470456 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e6290-5e5e-4477-9162-1dbf129b7bc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:57.470873 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e6290-5e5e-4477-9162-1dbf129b7bc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:57.471706 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3e6290-5e5e-4477-9162-1dbf129b7bc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:57.474629 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3e6290-5e5e-4477-9162-1dbf129b7bc0 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:57.502384 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 676/1370] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:32:57 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 31 12:32:57.507013 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3e6290-5e5e-4477-9162-1dbf129b7bc0 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:57.511069 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99971a3d-7888-404d-8d7e-d75843ce6099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:57.515520 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99971a3d-7888-404d-8d7e-d75843ce6099 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:57.516051 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99971a3d-7888-404d-8d7e-d75843ce6099 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:57.516590 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99971a3d-7888-404d-8d7e-d75843ce6099 admin admin] PATH_INFO: `/v3/users/5fcbe8cf40d747719937c4d264d4733e` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:57.518008 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99971a3d-7888-404d-8d7e-d75843ce6099 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5fcbe8cf40d747719937c4d264d4733e)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:57.525514 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3e6290-5e5e-4477-9162-1dbf129b7bc0 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:57.526709 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99971a3d-7888-404d-8d7e-d75843ce6099 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:57.542094 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c132ecd-cdf7-4431-a8ed-9aa472de5b90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:57.543031 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c132ecd-cdf7-4431-a8ed-9aa472de5b90 admin admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:57.543489 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c132ecd-cdf7-4431-a8ed-9aa472de5b90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:57.543938 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c132ecd-cdf7-4431-a8ed-9aa472de5b90 admin admin] PATH_INFO: `/v3/projects/80329fb711ad4e4ba5547af8f6f945e1/users/e4104a0759914252ba5d93394c38faf8/roles/176c07b4fe3b4537bdfeb4be21b379c5` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:57.548441 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c132ecd-cdf7-4431-a8ed-9aa472de5b90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80329fb711ad4e4ba5547af8f6f945e1, user_id=e4104a0759914252ba5d93394c38faf8, role_id=176c07b4fe3b4537bdfeb4be21b379c5)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:57.554829 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c132ecd-cdf7-4431-a8ed-9aa472de5b90 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:57.578140 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 677/1371] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:32:57 2022] PUT /identity/v3/projects/80329fb711ad4e4ba5547af8f6f945e1/users/e4104a0759914252ba5d93394c38faf8/roles/176c07b4fe3b4537bdfeb4be21b379c5 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 31 12:32:57.589558 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d6d3c8-6548-4bbf-88a3-53c30935916d None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:57.590112 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d6d3c8-6548-4bbf-88a3-53c30935916d None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:57.590529 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d6d3c8-6548-4bbf-88a3-53c30935916d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:57.637384 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:32:57.652545 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-99971a3d-7888-404d-8d7e-d75843ce6099 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5fcbe8cf40d747719937c4d264d4733e'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:32:57.666174 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-50d6d3c8-6548-4bbf-88a3-53c30935916d None None] MFA Rules not processed for user `e4104a0759914252ba5d93394c38faf8`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:32:57.670883 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-50d6d3c8-6548-4bbf-88a3-53c30935916d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:32:57.673454 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-99971a3d-7888-404d-8d7e-d75843ce6099 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5fcbe8cf40d747719937c4d264d4733e'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:32:57.674256 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-99971a3d-7888-404d-8d7e-d75843ce6099 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5fcbe8cf40d747719937c4d264d4733e'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:32:57.675954 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-99971a3d-7888-404d-8d7e-d75843ce6099 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5fcbe8cf40d747719937c4d264d4733e'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:32:57.708182 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-99971a3d-7888-404d-8d7e-d75843ce6099 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5fcbe8cf40d747719937c4d264d4733e'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:32:57.748313 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 695/1372] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:32:57 2022] DELETE /identity/v3/users/5fcbe8cf40d747719937c4d264d4733e => generated 0 bytes in 578 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:32:57.828017 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 678/1373] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:32:57 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 31 12:32:57.840547 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99971a3d-7888-404d-8d7e-d75843ce6099 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:57.842130 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-99971a3d-7888-404d-8d7e-d75843ce6099 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:32:57.871722 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:32:57.937659 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d6d3c8-6548-4bbf-88a3-53c30935916d None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:57.939141 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-50d6d3c8-6548-4bbf-88a3-53c30935916d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:32:58.048173 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99971a3d-7888-404d-8d7e-d75843ce6099 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:58.053313 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fad6522-b5a6-442e-9412-f0a117caf450 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d6d7ea077d49e8a66c6c6762499702', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:58.054268 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fad6522-b5a6-442e-9412-f0a117caf450 service cinder] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:58.054858 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fad6522-b5a6-442e-9412-f0a117caf450 service cinder] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:58.055331 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fad6522-b5a6-442e-9412-f0a117caf450 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:58.145349 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fad6522-b5a6-442e-9412-f0a117caf450 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:58.147578 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fad6522-b5a6-442e-9412-f0a117caf450 service cinder] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:58.167571 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 696/1374] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:32:57 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:32:58.177804 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d6d3c8-6548-4bbf-88a3-53c30935916d None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:58.238832 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fad6522-b5a6-442e-9412-f0a117caf450 service cinder] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:58.241160 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-4fad6522-b5a6-442e-9412-f0a117caf450 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:32:58.260095 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fad6522-b5a6-442e-9412-f0a117caf450 service cinder] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:58.286805 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35fc08a-b3f5-4453-b887-07f07e9123da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d6d7ea077d49e8a66c6c6762499702', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:58.290772 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35fc08a-b3f5-4453-b887-07f07e9123da service cinder] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:58.294554 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35fc08a-b3f5-4453-b887-07f07e9123da service cinder] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:58.295092 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35fc08a-b3f5-4453-b887-07f07e9123da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:58.301640 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-f35fc08a-b3f5-4453-b887-07f07e9123da service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:32:58.326200 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0723ff2-e500-4fad-9b9e-02b1db973d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:58.326865 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0723ff2-e500-4fad-9b9e-02b1db973d8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:58.327144 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0723ff2-e500-4fad-9b9e-02b1db973d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:58.327482 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0723ff2-e500-4fad-9b9e-02b1db973d8d admin admin] PATH_INFO: `/v3/users/729645f4a2c34c7d9a80b9e85773cbfd` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:58.336364 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0723ff2-e500-4fad-9b9e-02b1db973d8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=729645f4a2c34c7d9a80b9e85773cbfd)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:58.340581 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0723ff2-e500-4fad-9b9e-02b1db973d8d admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:58.492279 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:32:58.500924 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-e0723ff2-e500-4fad-9b9e-02b1db973d8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '729645f4a2c34c7d9a80b9e85773cbfd'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:32:58.522190 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-e0723ff2-e500-4fad-9b9e-02b1db973d8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '729645f4a2c34c7d9a80b9e85773cbfd'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:32:58.526165 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-e0723ff2-e500-4fad-9b9e-02b1db973d8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '729645f4a2c34c7d9a80b9e85773cbfd'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:32:58.536819 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35fc08a-b3f5-4453-b887-07f07e9123da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:58.538510 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35fc08a-b3f5-4453-b887-07f07e9123da service cinder] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:58.550652 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-e0723ff2-e500-4fad-9b9e-02b1db973d8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '729645f4a2c34c7d9a80b9e85773cbfd'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:32:58.580263 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-e0723ff2-e500-4fad-9b9e-02b1db973d8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '729645f4a2c34c7d9a80b9e85773cbfd'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:32:58.583977 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 697/1375] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:32:58 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 31 12:32:58.598894 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 679/1376] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:32:57 2022] DELETE /identity/v3/users/729645f4a2c34c7d9a80b9e85773cbfd => generated 0 bytes in 658 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 31 12:32:58.600620 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc21775-6b30-4626-8808-02491203bf42 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:58.601076 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc21775-6b30-4626-8808-02491203bf42 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:58.601519 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc21775-6b30-4626-8808-02491203bf42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:58.729957 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-7fc21775-6b30-4626-8808-02491203bf42 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:32:58.731536 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-7fc21775-6b30-4626-8808-02491203bf42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:32:58.953136 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:32:58.980763 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 698/1377] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:32:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:32:59.034138 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0723ff2-e500-4fad-9b9e-02b1db973d8d admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:59.041109 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-e0723ff2-e500-4fad-9b9e-02b1db973d8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:32:59.217824 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0723ff2-e500-4fad-9b9e-02b1db973d8d admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:59.233268 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130bb13d-89e5-4640-b16d-1d4a7b179696 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:59.236077 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130bb13d-89e5-4640-b16d-1d4a7b179696 service neutron] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:59.236995 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130bb13d-89e5-4640-b16d-1d4a7b179696 service neutron] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:59.238724 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130bb13d-89e5-4640-b16d-1d4a7b179696 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:59.267600 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130bb13d-89e5-4640-b16d-1d4a7b179696 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:59.271697 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130bb13d-89e5-4640-b16d-1d4a7b179696 service neutron] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:59.301975 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 680/1378] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:32:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:32:59.762555 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41370618-913a-4193-954c-8a20db248728 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:59.763242 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41370618-913a-4193-954c-8a20db248728 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:59.763778 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41370618-913a-4193-954c-8a20db248728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:59.802738 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130bb13d-89e5-4640-b16d-1d4a7b179696 service neutron] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:32:59.808474 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-130bb13d-89e5-4640-b16d-1d4a7b179696 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:32:59.826644 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-41370618-913a-4193-954c-8a20db248728 None None] MFA Rules not processed for user `70d59ee6f94848308665a32099cf5255`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:32:59.828364 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-41370618-913a-4193-954c-8a20db248728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:32:59.829500 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130bb13d-89e5-4640-b16d-1d4a7b179696 service neutron] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:32:59.847757 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a60f97-0d1f-403a-8db3-c39ff083253b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:32:59.848427 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a60f97-0d1f-403a-8db3-c39ff083253b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:59.848709 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a60f97-0d1f-403a-8db3-c39ff083253b admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:59.848982 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a60f97-0d1f-403a-8db3-c39ff083253b admin admin] PATH_INFO: `/v3/projects/8507a3a7b0fb4a21b3c528369a7ab8f9` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:59.850192 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a60f97-0d1f-403a-8db3-c39ff083253b admin admin] RBAC: Authorizing `identity:delete_project(project_id=8507a3a7b0fb4a21b3c528369a7ab8f9)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:32:59.854921 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a60f97-0d1f-403a-8db3-c39ff083253b admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:32:59.958741 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 699/1379] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:32:59 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:32:59.961166 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349976be-aaac-49fc-9761-2996bdbb514e None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:32:59.961705 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349976be-aaac-49fc-9761-2996bdbb514e None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:32:59.962612 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349976be-aaac-49fc-9761-2996bdbb514e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:32:59.974155 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-e9a60f97-0d1f-403a-8db3-c39ff083253b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8507a3a7b0fb4a21b3c528369a7ab8f9', 'request_id': 'req-e9a60f97-0d1f-403a-8db3-c39ff083253b'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:32:59.994440 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-e9a60f97-0d1f-403a-8db3-c39ff083253b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8507a3a7b0fb4a21b3c528369a7ab8f9', 'request_id': 'req-e9a60f97-0d1f-403a-8db3-c39ff083253b'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:00.014535 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-349976be-aaac-49fc-9761-2996bdbb514e None None] MFA Rules not processed for user `70d59ee6f94848308665a32099cf5255`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:33:00.017490 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-349976be-aaac-49fc-9761-2996bdbb514e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:00.021399 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-e9a60f97-0d1f-403a-8db3-c39ff083253b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8507a3a7b0fb4a21b3c528369a7ab8f9', 'request_id': 'req-e9a60f97-0d1f-403a-8db3-c39ff083253b'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:00.039812 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-e9a60f97-0d1f-403a-8db3-c39ff083253b admin admin] The token cache is being invalidate because project 8507a3a7b0fb4a21b3c528369a7ab8f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:33:00.040814 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-e9a60f97-0d1f-403a-8db3-c39ff083253b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:00.043904 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-e9a60f97-0d1f-403a-8db3-c39ff083253b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:00.047025 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 681/1380] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:32:59 2022] DELETE /identity/v3/projects/8507a3a7b0fb4a21b3c528369a7ab8f9 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:33:00.055029 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f805cf-28a9-4800-a302-0e196f777e5e None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:00.055406 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f805cf-28a9-4800-a302-0e196f777e5e None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:00.055691 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f805cf-28a9-4800-a302-0e196f777e5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:00.095302 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-d2f805cf-28a9-4800-a302-0e196f777e5e None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:33:00.097088 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-d2f805cf-28a9-4800-a302-0e196f777e5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:00.282611 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:33:00.297414 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 700/1381] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:32:59 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:33:00.309417 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349976be-aaac-49fc-9761-2996bdbb514e None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:00.313080 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-349976be-aaac-49fc-9761-2996bdbb514e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:00.356810 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 682/1382] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:33:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 31 12:33:00.358307 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824c9b4e-ba77-4341-8622-56f8bc277e5e None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:00.358578 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824c9b4e-ba77-4341-8622-56f8bc277e5e None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:00.358845 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824c9b4e-ba77-4341-8622-56f8bc277e5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:00.404320 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-824c9b4e-ba77-4341-8622-56f8bc277e5e None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:33:00.412651 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-824c9b4e-ba77-4341-8622-56f8bc277e5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:00.431469 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 683/1383] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:33:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:33:00.436549 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824c9b4e-ba77-4341-8622-56f8bc277e5e None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:00.453440 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824c9b4e-ba77-4341-8622-56f8bc277e5e None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:00.461242 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa00553-b478-4f3b-91c1-d4e320533698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:00.462473 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa00553-b478-4f3b-91c1-d4e320533698 admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:00.462915 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa00553-b478-4f3b-91c1-d4e320533698 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:00.463393 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa00553-b478-4f3b-91c1-d4e320533698 admin admin] PATH_INFO: `/v3/domains` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:00.464214 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa00553-b478-4f3b-91c1-d4e320533698 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:33:00.464673 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa00553-b478-4f3b-91c1-d4e320533698 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:00.466952 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa00553-b478-4f3b-91c1-d4e320533698 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:00.491797 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 684/1384] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Oct 31 12:33:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:33:00.493090 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa00553-b478-4f3b-91c1-d4e320533698 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:00.495385 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349976be-aaac-49fc-9761-2996bdbb514e None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:00.509773 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fc371c-4b1d-4521-9ad1-ecf31dc02b27 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99889b1dab0b471d9c77887cffefc652', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:00.509919 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa00553-b478-4f3b-91c1-d4e320533698 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:00.510418 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fc371c-4b1d-4521-9ad1-ecf31dc02b27 service nova] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:00.544375 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fc371c-4b1d-4521-9ad1-ecf31dc02b27 service nova] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:00.544375 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fc371c-4b1d-4521-9ad1-ecf31dc02b27 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:00.544375 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-84fc371c-4b1d-4521-9ad1-ecf31dc02b27 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:00.549949 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a6de1f-bcac-483a-a312-8a3633c69a57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:00.549949 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a6de1f-bcac-483a-a312-8a3633c69a57 admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:00.549949 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a6de1f-bcac-483a-a312-8a3633c69a57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:00.549949 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a6de1f-bcac-483a-a312-8a3633c69a57 admin admin] PATH_INFO: `/v3/projects` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:00.555760 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a6de1f-bcac-483a-a312-8a3633c69a57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:00.575943 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a6de1f-bcac-483a-a312-8a3633c69a57 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:00.595474 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fc371c-4b1d-4521-9ad1-ecf31dc02b27 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:00.597386 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fc371c-4b1d-4521-9ad1-ecf31dc02b27 service nova] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:00.638884 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 701/1385] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:33:00 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 31 12:33:00.664424 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 685/1386] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:33:00 2022] POST /identity/v3/projects => generated 387 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:33:00.665616 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fc371c-4b1d-4521-9ad1-ecf31dc02b27 service nova] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:00.691549 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fc371c-4b1d-4521-9ad1-ecf31dc02b27 service nova] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:00.751448 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:33:00.938069 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae92687-7016-4b3f-9079-400b612d0b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:00.938930 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae92687-7016-4b3f-9079-400b612d0b15 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:00.939440 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae92687-7016-4b3f-9079-400b612d0b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:00.939906 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae92687-7016-4b3f-9079-400b612d0b15 admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:00.941282 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae92687-7016-4b3f-9079-400b612d0b15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:00.943677 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae92687-7016-4b3f-9079-400b612d0b15 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:01.004003 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 702/1387] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:33:00 2022] POST /identity/v3/users => generated 294 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:33:01.010003 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a6de1f-bcac-483a-a312-8a3633c69a57 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:01.031328 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a6de1f-bcac-483a-a312-8a3633c69a57 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:01.058279 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95df5e5-50bf-465c-9e9a-e97d2bbc4fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:01.060012 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95df5e5-50bf-465c-9e9a-e97d2bbc4fce admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:01.060593 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95df5e5-50bf-465c-9e9a-e97d2bbc4fce admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:01.061158 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95df5e5-50bf-465c-9e9a-e97d2bbc4fce admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:01.063597 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95df5e5-50bf-465c-9e9a-e97d2bbc4fce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:01.067082 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95df5e5-50bf-465c-9e9a-e97d2bbc4fce admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:01.092135 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 686/1388] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:33:01 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:33:01.095345 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae92687-7016-4b3f-9079-400b612d0b15 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:01.111341 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae92687-7016-4b3f-9079-400b612d0b15 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:01.121913 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec32cccd-e867-4f92-a676-aed82ae15c14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:01.123333 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec32cccd-e867-4f92-a676-aed82ae15c14 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:01.124501 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec32cccd-e867-4f92-a676-aed82ae15c14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:01.124969 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec32cccd-e867-4f92-a676-aed82ae15c14 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:01.125774 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec32cccd-e867-4f92-a676-aed82ae15c14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:01.130805 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec32cccd-e867-4f92-a676-aed82ae15c14 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:01.167953 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 703/1389] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:33:01 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:33:01.169312 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95df5e5-50bf-465c-9e9a-e97d2bbc4fce admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:01.191422 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95df5e5-50bf-465c-9e9a-e97d2bbc4fce admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:01.218239 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7208d5-7444-4561-9452-f0a0145b89e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:01.228712 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7208d5-7444-4561-9452-f0a0145b89e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:01.237356 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7208d5-7444-4561-9452-f0a0145b89e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:01.242710 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7208d5-7444-4561-9452-f0a0145b89e8 admin admin] PATH_INFO: `/v3/projects/8ae36f67568d4091b640ef57ce7e1e51/users/2361fb48804545418cc390c55d10a521/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:01.301045 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7208d5-7444-4561-9452-f0a0145b89e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ae36f67568d4091b640ef57ce7e1e51, user_id=2361fb48804545418cc390c55d10a521, role_id=ca1a5695f8bd499aa0b8cedaeecd2cc6)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:01.307859 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7208d5-7444-4561-9452-f0a0145b89e8 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:01.375694 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 687/1390] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:33:01 2022] PUT /identity/v3/projects/8ae36f67568d4091b640ef57ce7e1e51/users/2361fb48804545418cc390c55d10a521/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:33:01.387701 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36202a1-5bf5-479a-a430-8620132c4864 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:01.388531 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36202a1-5bf5-479a-a430-8620132c4864 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:01.388531 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36202a1-5bf5-479a-a430-8620132c4864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:01.460701 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-b36202a1-5bf5-479a-a430-8620132c4864 None None] MFA Rules not processed for user `2361fb48804545418cc390c55d10a521`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:33:01.464485 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-b36202a1-5bf5-479a-a430-8620132c4864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:01.562505 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d94fd29-10b7-4000-a783-f02f156fa6a0 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:01.563178 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d94fd29-10b7-4000-a783-f02f156fa6a0 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:01.563647 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d94fd29-10b7-4000-a783-f02f156fa6a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:01.655413 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-0d94fd29-10b7-4000-a783-f02f156fa6a0 None None] MFA Rules not processed for user `db2e978bcb824a5e9d593b861c9e3bd7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:33:01.657115 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-0d94fd29-10b7-4000-a783-f02f156fa6a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:01.674263 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 704/1391] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:33:01 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:33:01.677497 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36202a1-5bf5-479a-a430-8620132c4864 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:01.856822 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 688/1392] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:33:01 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 31 12:33:01.867022 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50252332-77ac-4523-8439-2790f0624b28 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:01.867624 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50252332-77ac-4523-8439-2790f0624b28 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:01.868092 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50252332-77ac-4523-8439-2790f0624b28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:01.922575 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-50252332-77ac-4523-8439-2790f0624b28 None None] MFA Rules not processed for user `db2e978bcb824a5e9d593b861c9e3bd7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:33:01.925777 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-50252332-77ac-4523-8439-2790f0624b28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:01.946184 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 689/1393] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:33:01 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 31 12:33:01.959773 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50252332-77ac-4523-8439-2790f0624b28 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:01.962957 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-50252332-77ac-4523-8439-2790f0624b28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:01.991628 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36202a1-5bf5-479a-a430-8620132c4864 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:01.998486 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27399791-3dd2-4b32-b3aa-1b30bad02cbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:01.999388 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27399791-3dd2-4b32-b3aa-1b30bad02cbd admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:01.999740 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27399791-3dd2-4b32-b3aa-1b30bad02cbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:02.000769 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27399791-3dd2-4b32-b3aa-1b30bad02cbd admin admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:02.001504 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27399791-3dd2-4b32-b3aa-1b30bad02cbd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:02.004264 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27399791-3dd2-4b32-b3aa-1b30bad02cbd admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:02.063985 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 705/1394] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:33:01 2022] POST /identity/v3/projects => generated 385 bytes in 389 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:33:02.068389 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27399791-3dd2-4b32-b3aa-1b30bad02cbd admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:02.096457 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:33:02.105697 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:33:02.155387 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50252332-77ac-4523-8439-2790f0624b28 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:02.276325 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27399791-3dd2-4b32-b3aa-1b30bad02cbd admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:02.285531 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29ed10d-e5d5-4279-afb6-ce4b1d3a4901 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99889b1dab0b471d9c77887cffefc652', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:02.286404 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29ed10d-e5d5-4279-afb6-ce4b1d3a4901 service nova] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:02.286701 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29ed10d-e5d5-4279-afb6-ce4b1d3a4901 service nova] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:02.286973 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29ed10d-e5d5-4279-afb6-ce4b1d3a4901 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:02.294906 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0b9ba8-32ea-422a-b81d-1f18c423a928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:02.295837 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0b9ba8-32ea-422a-b81d-1f18c423a928 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:02.296312 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0b9ba8-32ea-422a-b81d-1f18c423a928 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:02.296835 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0b9ba8-32ea-422a-b81d-1f18c423a928 admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:02.298307 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0b9ba8-32ea-422a-b81d-1f18c423a928 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:02.303098 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0b9ba8-32ea-422a-b81d-1f18c423a928 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:02.358146 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 706/1395] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:33:02 2022] POST /identity/v3/users => generated 299 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 31 12:33:02.359254 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0b9ba8-32ea-422a-b81d-1f18c423a928 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:02.366814 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-0d0b9ba8-32ea-422a-b81d-1f18c423a928 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:02.392260 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29ed10d-e5d5-4279-afb6-ce4b1d3a4901 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:02.396057 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29ed10d-e5d5-4279-afb6-ce4b1d3a4901 service nova] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:02.404440 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0b9ba8-32ea-422a-b81d-1f18c423a928 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:02.419076 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9792d3d-3962-485b-83a0-2ba33c428d35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:02.419784 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9792d3d-3962-485b-83a0-2ba33c428d35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:02.420064 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9792d3d-3962-485b-83a0-2ba33c428d35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:02.420494 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9792d3d-3962-485b-83a0-2ba33c428d35 admin admin] PATH_INFO: `/v3/projects/f7c7ae7636c54e8d93df903ff10551df` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:02.422937 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9792d3d-3962-485b-83a0-2ba33c428d35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7c7ae7636c54e8d93df903ff10551df)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:02.429663 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9792d3d-3962-485b-83a0-2ba33c428d35 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:02.435882 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 690/1396] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:33:01 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:33:02.443209 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc0a60-ac73-4423-a997-598be88e7083 None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:02.443524 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc0a60-ac73-4423-a997-598be88e7083 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:02.443806 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc0a60-ac73-4423-a997-598be88e7083 None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:02.450676 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 691/1397] 10.222.0.15 () {54 vars in 912 bytes} [Mon Oct 31 12:33:02 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:33:02.452415 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abc0a60-ac73-4423-a997-598be88e7083 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:02.513177 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abc0a60-ac73-4423-a997-598be88e7083 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:02.540750 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-f9792d3d-3962-485b-83a0-2ba33c428d35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7c7ae7636c54e8d93df903ff10551df', 'request_id': 'req-f9792d3d-3962-485b-83a0-2ba33c428d35'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:02.556699 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-f9792d3d-3962-485b-83a0-2ba33c428d35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7c7ae7636c54e8d93df903ff10551df', 'request_id': 'req-f9792d3d-3962-485b-83a0-2ba33c428d35'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:02.578981 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-f9792d3d-3962-485b-83a0-2ba33c428d35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7c7ae7636c54e8d93df903ff10551df', 'request_id': 'req-f9792d3d-3962-485b-83a0-2ba33c428d35'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:02.604507 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-f9792d3d-3962-485b-83a0-2ba33c428d35 admin admin] The token cache is being invalidate because project f7c7ae7636c54e8d93df903ff10551df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:33:02.604811 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-f9792d3d-3962-485b-83a0-2ba33c428d35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:02.606980 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-f9792d3d-3962-485b-83a0-2ba33c428d35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:02.610702 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 707/1398] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:33:02 2022] DELETE /identity/v3/projects/f7c7ae7636c54e8d93df903ff10551df => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:33:02.614470 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393ec9b3-f8b1-4bad-8286-97493b7b358c None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:02.614739 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393ec9b3-f8b1-4bad-8286-97493b7b358c None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:02.615322 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393ec9b3-f8b1-4bad-8286-97493b7b358c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:02.776644 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-393ec9b3-f8b1-4bad-8286-97493b7b358c None None] MFA Rules not processed for user `25e9c7d1e4714bf18a1ced84855423d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:33:02.785149 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-393ec9b3-f8b1-4bad-8286-97493b7b358c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:02.804033 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62183a60-230b-4347-99c7-bb17d09e4d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:02.805897 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62183a60-230b-4347-99c7-bb17d09e4d1b admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:02.811277 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62183a60-230b-4347-99c7-bb17d09e4d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:02.811722 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62183a60-230b-4347-99c7-bb17d09e4d1b admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:02.812743 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62183a60-230b-4347-99c7-bb17d09e4d1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:02.814467 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62183a60-230b-4347-99c7-bb17d09e4d1b admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:02.837749 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 692/1399] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:33:02 2022] GET /identity/v3/roles => generated 1420 bytes in 384 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:33:02.837749 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62183a60-230b-4347-99c7-bb17d09e4d1b admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:02.843575 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-62183a60-230b-4347-99c7-bb17d09e4d1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:02.869382 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62183a60-230b-4347-99c7-bb17d09e4d1b admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:02.881517 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebea7b1f-b7c9-4e60-a012-54acea43f7c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:02.882286 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebea7b1f-b7c9-4e60-a012-54acea43f7c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:02.882675 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebea7b1f-b7c9-4e60-a012-54acea43f7c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:02.883055 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebea7b1f-b7c9-4e60-a012-54acea43f7c0 admin admin] PATH_INFO: `/v3/users/c358bd97a32f4ddb96dc0e71d4dc1209` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:02.894863 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebea7b1f-b7c9-4e60-a012-54acea43f7c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c358bd97a32f4ddb96dc0e71d4dc1209)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:02.897421 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebea7b1f-b7c9-4e60-a012-54acea43f7c0 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:02.937830 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:33:03.002823 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-ebea7b1f-b7c9-4e60-a012-54acea43f7c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c358bd97a32f4ddb96dc0e71d4dc1209'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:03.036881 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-ebea7b1f-b7c9-4e60-a012-54acea43f7c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c358bd97a32f4ddb96dc0e71d4dc1209'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:03.037998 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-ebea7b1f-b7c9-4e60-a012-54acea43f7c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c358bd97a32f4ddb96dc0e71d4dc1209'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:03.047481 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 708/1400] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Oct 31 12:33:02 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 436 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:33:03.054156 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393ec9b3-f8b1-4bad-8286-97493b7b358c None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:03.059061 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-393ec9b3-f8b1-4bad-8286-97493b7b358c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:03.091886 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-ebea7b1f-b7c9-4e60-a012-54acea43f7c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c358bd97a32f4ddb96dc0e71d4dc1209'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:03.104692 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393ec9b3-f8b1-4bad-8286-97493b7b358c None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:03.115313 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-ebea7b1f-b7c9-4e60-a012-54acea43f7c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c358bd97a32f4ddb96dc0e71d4dc1209'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:03.119913 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c843606a-ef68-4574-b9d5-3cb80daaa88d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:03.121643 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c843606a-ef68-4574-b9d5-3cb80daaa88d admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:03.122736 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c843606a-ef68-4574-b9d5-3cb80daaa88d admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:03.123864 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c843606a-ef68-4574-b9d5-3cb80daaa88d admin admin] PATH_INFO: `/v3/projects/82311da4f89c402a854f67e5aade87da/users/50c789f1583f404d9175186457f14231/roles/176c07b4fe3b4537bdfeb4be21b379c5` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:03.139717 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 693/1401] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:33:02 2022] DELETE /identity/v3/users/c358bd97a32f4ddb96dc0e71d4dc1209 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:33:03.141299 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebea7b1f-b7c9-4e60-a012-54acea43f7c0 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:03.145953 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-ebea7b1f-b7c9-4e60-a012-54acea43f7c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:03.166918 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c843606a-ef68-4574-b9d5-3cb80daaa88d admin admin] RBAC: Authorizing `identity:create_grant(project_id=82311da4f89c402a854f67e5aade87da, user_id=50c789f1583f404d9175186457f14231, role_id=176c07b4fe3b4537bdfeb4be21b379c5)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:03.174501 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c843606a-ef68-4574-b9d5-3cb80daaa88d admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:03.198838 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:33:03.209995 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 709/1402] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:33:03 2022] PUT /identity/v3/projects/82311da4f89c402a854f67e5aade87da/users/50c789f1583f404d9175186457f14231/roles/176c07b4fe3b4537bdfeb4be21b379c5 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:33:03.213863 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c843606a-ef68-4574-b9d5-3cb80daaa88d admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:03.218057 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-c843606a-ef68-4574-b9d5-3cb80daaa88d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:03.357294 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebea7b1f-b7c9-4e60-a012-54acea43f7c0 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:03.405905 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c843606a-ef68-4574-b9d5-3cb80daaa88d admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:03.425953 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d5a70a-3197-4fb6-9ff7-61b84aa0d305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:03.426588 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d5a70a-3197-4fb6-9ff7-61b84aa0d305 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:03.426867 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d5a70a-3197-4fb6-9ff7-61b84aa0d305 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:03.427258 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d5a70a-3197-4fb6-9ff7-61b84aa0d305 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:03.427903 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d5a70a-3197-4fb6-9ff7-61b84aa0d305 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:03.429520 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d5a70a-3197-4fb6-9ff7-61b84aa0d305 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:03.453493 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 710/1403] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:33:03 2022] GET /identity/v3/roles => generated 1420 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 31 12:33:03.454706 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb27ecf5-477a-405c-9cdf-b223a3fbe383 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:03.455037 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb27ecf5-477a-405c-9cdf-b223a3fbe383 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:03.457491 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb27ecf5-477a-405c-9cdf-b223a3fbe383 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:03.463881 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 711/1404] 10.222.0.15 () {54 vars in 912 bytes} [Mon Oct 31 12:33:03 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:33:03.464641 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb27ecf5-477a-405c-9cdf-b223a3fbe383 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:03.498066 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f589afe9-27c3-4fd6-8aab-9dac7afe831a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:03.499198 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f589afe9-27c3-4fd6-8aab-9dac7afe831a service neutron] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:03.499739 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f589afe9-27c3-4fd6-8aab-9dac7afe831a service neutron] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:03.499860 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb27ecf5-477a-405c-9cdf-b223a3fbe383 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:03.500193 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f589afe9-27c3-4fd6-8aab-9dac7afe831a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:03.503691 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-f589afe9-27c3-4fd6-8aab-9dac7afe831a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:03.514727 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebfaab0-0cfe-4ecc-9c8b-2599c7921b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:03.515433 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebfaab0-0cfe-4ecc-9c8b-2599c7921b43 admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:03.515710 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebfaab0-0cfe-4ecc-9c8b-2599c7921b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:03.515978 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebfaab0-0cfe-4ecc-9c8b-2599c7921b43 admin admin] PATH_INFO: `/v3/projects/82311da4f89c402a854f67e5aade87da/users/50c789f1583f404d9175186457f14231/roles/176c07b4fe3b4537bdfeb4be21b379c5` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:03.517927 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebfaab0-0cfe-4ecc-9c8b-2599c7921b43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82311da4f89c402a854f67e5aade87da, user_id=50c789f1583f404d9175186457f14231, role_id=176c07b4fe3b4537bdfeb4be21b379c5)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:03.521115 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebfaab0-0cfe-4ecc-9c8b-2599c7921b43 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:03.527944 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f589afe9-27c3-4fd6-8aab-9dac7afe831a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:03.530130 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f589afe9-27c3-4fd6-8aab-9dac7afe831a service neutron] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:03.538881 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 712/1405] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:33:03 2022] PUT /identity/v3/projects/82311da4f89c402a854f67e5aade87da/users/50c789f1583f404d9175186457f14231/roles/176c07b4fe3b4537bdfeb4be21b379c5 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:33:03.540852 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e069ef-f6ba-4aaf-85d8-521758c33307 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:03.541145 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e069ef-f6ba-4aaf-85d8-521758c33307 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:03.541416 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e069ef-f6ba-4aaf-85d8-521758c33307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:03.569089 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 694/1406] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:33:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:33:03.571097 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24975172-d3e2-4003-a5a7-212f0d71cf08 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:03.571414 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24975172-d3e2-4003-a5a7-212f0d71cf08 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:03.571680 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24975172-d3e2-4003-a5a7-212f0d71cf08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:03.645260 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-f4e069ef-f6ba-4aaf-85d8-521758c33307 None None] MFA Rules not processed for user `25e9c7d1e4714bf18a1ced84855423d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:33:03.646733 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-f4e069ef-f6ba-4aaf-85d8-521758c33307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:03.653644 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-24975172-d3e2-4003-a5a7-212f0d71cf08 None None] MFA Rules not processed for user `50c789f1583f404d9175186457f14231`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:33:03.655510 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-24975172-d3e2-4003-a5a7-212f0d71cf08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:03.761513 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 713/1407] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Oct 31 12:33:03 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:33:03.763606 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9f014a-1d28-42db-a387-e0086a0f2017 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:03.764163 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9f014a-1d28-42db-a387-e0086a0f2017 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:03.764716 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9f014a-1d28-42db-a387-e0086a0f2017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:03.810085 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-5f9f014a-1d28-42db-a387-e0086a0f2017 None None] MFA Rules not processed for user `d6c1beb321e44908ab987b17f7b74484`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:33:03.811153 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:33:03.819965 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-5f9f014a-1d28-42db-a387-e0086a0f2017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:03.901688 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 695/1408] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:33:03 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 332 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:33:03.902637 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24975172-d3e2-4003-a5a7-212f0d71cf08 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:03.904348 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-24975172-d3e2-4003-a5a7-212f0d71cf08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:03.921287 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 714/1409] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:33:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:33:03.928525 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9f014a-1d28-42db-a387-e0086a0f2017 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:03.930150 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-5f9f014a-1d28-42db-a387-e0086a0f2017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:03.933222 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24975172-d3e2-4003-a5a7-212f0d71cf08 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:03.942544 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a392a010-8e3c-4a89-b8f5-2637be94a9b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:03.943280 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a392a010-8e3c-4a89-b8f5-2637be94a9b0 service neutron] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:03.943546 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a392a010-8e3c-4a89-b8f5-2637be94a9b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:03.943817 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a392a010-8e3c-4a89-b8f5-2637be94a9b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:03.963500 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a392a010-8e3c-4a89-b8f5-2637be94a9b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:03.964966 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a392a010-8e3c-4a89-b8f5-2637be94a9b0 service neutron] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:03.989292 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 696/1410] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:33:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:33:04.033232 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:33:04.079127 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9f014a-1d28-42db-a387-e0086a0f2017 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:04.087143 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c37373-ab5b-47ff-883f-e3456c628a2e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d6d7ea077d49e8a66c6c6762499702', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:04.088110 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c37373-ab5b-47ff-883f-e3456c628a2e service cinder] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:04.088520 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c37373-ab5b-47ff-883f-e3456c628a2e service cinder] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:04.088934 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c37373-ab5b-47ff-883f-e3456c628a2e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:04.124138 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c37373-ab5b-47ff-883f-e3456c628a2e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:04.125720 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c37373-ab5b-47ff-883f-e3456c628a2e service cinder] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:04.137355 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a392a010-8e3c-4a89-b8f5-2637be94a9b0 service neutron] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:04.139297 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-a392a010-8e3c-4a89-b8f5-2637be94a9b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:04.142688 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 715/1411] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:33:03 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:33:04.180296 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c37373-ab5b-47ff-883f-e3456c628a2e service cinder] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:04.269476 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a392a010-8e3c-4a89-b8f5-2637be94a9b0 service neutron] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:04.282280 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5855d78c-a59a-421b-ad6e-fbf3d72e3d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:04.283216 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5855d78c-a59a-421b-ad6e-fbf3d72e3d16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:04.283673 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5855d78c-a59a-421b-ad6e-fbf3d72e3d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:04.284119 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5855d78c-a59a-421b-ad6e-fbf3d72e3d16 admin admin] PATH_INFO: `/v3/users/62bdb7d0f7484cc79c1c02fe1b79c671` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:04.285765 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5855d78c-a59a-421b-ad6e-fbf3d72e3d16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=62bdb7d0f7484cc79c1c02fe1b79c671)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:04.304971 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5855d78c-a59a-421b-ad6e-fbf3d72e3d16 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:04.307766 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c37373-ab5b-47ff-883f-e3456c628a2e service cinder] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:04.361820 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08887dd1-c37b-4c7e-ba4b-603305aeeb6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:04.364857 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08887dd1-c37b-4c7e-ba4b-603305aeeb6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:04.367503 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08887dd1-c37b-4c7e-ba4b-603305aeeb6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:04.367928 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08887dd1-c37b-4c7e-ba4b-603305aeeb6f admin admin] PATH_INFO: `/v3/users/2361fb48804545418cc390c55d10a521` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:04.371297 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08887dd1-c37b-4c7e-ba4b-603305aeeb6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2361fb48804545418cc390c55d10a521)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:04.374619 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08887dd1-c37b-4c7e-ba4b-603305aeeb6f admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:04.472864 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-5855d78c-a59a-421b-ad6e-fbf3d72e3d16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62bdb7d0f7484cc79c1c02fe1b79c671'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:04.486913 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-5855d78c-a59a-421b-ad6e-fbf3d72e3d16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62bdb7d0f7484cc79c1c02fe1b79c671'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:04.490202 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-5855d78c-a59a-421b-ad6e-fbf3d72e3d16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62bdb7d0f7484cc79c1c02fe1b79c671'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:04.537328 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-5855d78c-a59a-421b-ad6e-fbf3d72e3d16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62bdb7d0f7484cc79c1c02fe1b79c671'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:04.551586 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-08887dd1-c37b-4c7e-ba4b-603305aeeb6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2361fb48804545418cc390c55d10a521'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:04.562546 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-5855d78c-a59a-421b-ad6e-fbf3d72e3d16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62bdb7d0f7484cc79c1c02fe1b79c671'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:04.568713 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-08887dd1-c37b-4c7e-ba4b-603305aeeb6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2361fb48804545418cc390c55d10a521'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:04.569377 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-08887dd1-c37b-4c7e-ba4b-603305aeeb6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2361fb48804545418cc390c55d10a521'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:04.570036 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-08887dd1-c37b-4c7e-ba4b-603305aeeb6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2361fb48804545418cc390c55d10a521'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:04.573705 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 697/1412] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:33:04 2022] DELETE /identity/v3/users/62bdb7d0f7484cc79c1c02fe1b79c671 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 31 12:33:04.580573 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e50223-b60e-4e67-8ab5-57adb4c368ce None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:04.583027 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e50223-b60e-4e67-8ab5-57adb4c368ce None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:04.584205 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e50223-b60e-4e67-8ab5-57adb4c368ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:04.592077 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-08887dd1-c37b-4c7e-ba4b-603305aeeb6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2361fb48804545418cc390c55d10a521'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:04.617267 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 716/1413] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:33:04 2022] DELETE /identity/v3/users/2361fb48804545418cc390c55d10a521 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:33:04.622129 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0503d47-e632-4dd7-b5ca-a815db29a635 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:04.622600 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0503d47-e632-4dd7-b5ca-a815db29a635 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:04.622882 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0503d47-e632-4dd7-b5ca-a815db29a635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:04.635988 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-d7e50223-b60e-4e67-8ab5-57adb4c368ce None None] MFA Rules not processed for user `a6d1ace68aa84499a4929da60f3d71bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:33:04.653056 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-d7e50223-b60e-4e67-8ab5-57adb4c368ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:04.664313 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-f0503d47-e632-4dd7-b5ca-a815db29a635 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:33:04.665688 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-f0503d47-e632-4dd7-b5ca-a815db29a635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:04.734531 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:33:04.798301 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:33:04.829067 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 717/1414] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:33:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:33:04.834314 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e36f9c-0909-494a-8ab6-ba476e9552bd None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:04.834624 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e36f9c-0909-494a-8ab6-ba476e9552bd None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:04.835259 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e36f9c-0909-494a-8ab6-ba476e9552bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:04.869408 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 698/1415] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:33:04 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:33:04.870482 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e50223-b60e-4e67-8ab5-57adb4c368ce None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:04.880392 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-d7e50223-b60e-4e67-8ab5-57adb4c368ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:04.906580 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-65e36f9c-0909-494a-8ab6-ba476e9552bd None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:33:04.907844 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-65e36f9c-0909-494a-8ab6-ba476e9552bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:04.926808 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 718/1416] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:33:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:33:04.939183 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e36f9c-0909-494a-8ab6-ba476e9552bd None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:04.941194 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-65e36f9c-0909-494a-8ab6-ba476e9552bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:05.052659 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e50223-b60e-4e67-8ab5-57adb4c368ce None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:05.057054 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431898d5-3911-4778-872d-5bf84d88e2df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:05.058056 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431898d5-3911-4778-872d-5bf84d88e2df service neutron] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:05.058411 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431898d5-3911-4778-872d-5bf84d88e2df service neutron] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:05.058886 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431898d5-3911-4778-872d-5bf84d88e2df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:05.071382 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431898d5-3911-4778-872d-5bf84d88e2df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:05.073230 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431898d5-3911-4778-872d-5bf84d88e2df service neutron] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:05.088514 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 699/1417] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:33:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:33:05.089449 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431898d5-3911-4778-872d-5bf84d88e2df service neutron] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:05.090879 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-431898d5-3911-4778-872d-5bf84d88e2df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:05.096210 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e36f9c-0909-494a-8ab6-ba476e9552bd None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:05.109060 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f58338d-33cf-42d4-8ec1-3f1857db53e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:05.110112 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f58338d-33cf-42d4-8ec1-3f1857db53e5 service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:05.110396 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f58338d-33cf-42d4-8ec1-3f1857db53e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:05.110661 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f58338d-33cf-42d4-8ec1-3f1857db53e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:05.114008 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431898d5-3911-4778-872d-5bf84d88e2df service neutron] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:05.125777 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e383ece-7408-49ae-b0b5-f226b12449e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:05.127895 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e383ece-7408-49ae-b0b5-f226b12449e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:05.128750 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e383ece-7408-49ae-b0b5-f226b12449e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:05.129780 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e383ece-7408-49ae-b0b5-f226b12449e3 admin admin] PATH_INFO: `/v3/projects/17fde387d0a94493b34bb5b20ee9aa4c` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:05.133759 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e383ece-7408-49ae-b0b5-f226b12449e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=17fde387d0a94493b34bb5b20ee9aa4c)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:05.134044 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f58338d-33cf-42d4-8ec1-3f1857db53e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:05.135544 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f58338d-33cf-42d4-8ec1-3f1857db53e5 service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:05.138839 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e383ece-7408-49ae-b0b5-f226b12449e3 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:05.161501 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 719/1418] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:33:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:33:05.162359 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f58338d-33cf-42d4-8ec1-3f1857db53e5 service neutron] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:05.163720 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-5f58338d-33cf-42d4-8ec1-3f1857db53e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:05.265583 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-3e383ece-7408-49ae-b0b5-f226b12449e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17fde387d0a94493b34bb5b20ee9aa4c', 'request_id': 'req-3e383ece-7408-49ae-b0b5-f226b12449e3'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:05.275874 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-3e383ece-7408-49ae-b0b5-f226b12449e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17fde387d0a94493b34bb5b20ee9aa4c', 'request_id': 'req-3e383ece-7408-49ae-b0b5-f226b12449e3'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:05.293374 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-3e383ece-7408-49ae-b0b5-f226b12449e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17fde387d0a94493b34bb5b20ee9aa4c', 'request_id': 'req-3e383ece-7408-49ae-b0b5-f226b12449e3'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:05.326447 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f58338d-33cf-42d4-8ec1-3f1857db53e5 service neutron] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:05.332907 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-3e383ece-7408-49ae-b0b5-f226b12449e3 admin admin] The token cache is being invalidate because project 17fde387d0a94493b34bb5b20ee9aa4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:33:05.332907 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-3e383ece-7408-49ae-b0b5-f226b12449e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:05.334116 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-3e383ece-7408-49ae-b0b5-f226b12449e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:05.339563 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 700/1419] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:33:05 2022] DELETE /identity/v3/projects/17fde387d0a94493b34bb5b20ee9aa4c => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:33:05.347618 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0e784e-ce60-42df-85b7-0dcc124a979c None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:05.348295 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0e784e-ce60-42df-85b7-0dcc124a979c None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:05.349380 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0e784e-ce60-42df-85b7-0dcc124a979c None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:05.358468 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 701/1420] 10.222.0.15 () {54 vars in 961 bytes} [Mon Oct 31 12:33:05 2022] GET /identity => generated 270 bytes in 18 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:33:05.361471 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db63303-7e12-4a2f-9c90-3b8fa23407e9 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:05.362979 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db63303-7e12-4a2f-9c90-3b8fa23407e9 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:05.364681 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db63303-7e12-4a2f-9c90-3b8fa23407e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:05.413499 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-0db63303-7e12-4a2f-9c90-3b8fa23407e9 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:33:05.418198 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-0db63303-7e12-4a2f-9c90-3b8fa23407e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:05.501218 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e0f07c-4ba9-4e61-b800-48ab9287a64a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e7777495d7a406c8c50b1ad52e47263', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:05.501861 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e0f07c-4ba9-4e61-b800-48ab9287a64a service placement] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:05.502138 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e0f07c-4ba9-4e61-b800-48ab9287a64a service placement] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:05.502408 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e0f07c-4ba9-4e61-b800-48ab9287a64a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:05.509366 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-84e0f07c-4ba9-4e61-b800-48ab9287a64a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:05.538445 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e0f07c-4ba9-4e61-b800-48ab9287a64a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:05.542990 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e0f07c-4ba9-4e61-b800-48ab9287a64a service placement] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:05.587588 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 720/1421] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:33:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:33:05.592363 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eff4aba-d584-4140-a936-1563c67769b8 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:05.592599 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eff4aba-d584-4140-a936-1563c67769b8 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:05.595229 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 702/1422] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:33:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:33:05.595412 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eff4aba-d584-4140-a936-1563c67769b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:05.610936 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9c2064-3319-4dc6-a9bd-6eb8342197d9 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:05.611280 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9c2064-3319-4dc6-a9bd-6eb8342197d9 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:05.614488 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9c2064-3319-4dc6-a9bd-6eb8342197d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:05.644273 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:33:05.702829 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-0eff4aba-d584-4140-a936-1563c67769b8 None None] MFA Rules not processed for user `9e7777495d7a406c8c50b1ad52e47263`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:33:05.707708 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-0eff4aba-d584-4140-a936-1563c67769b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:05.709256 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-5a9c2064-3319-4dc6-a9bd-6eb8342197d9 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:33:05.712279 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-5a9c2064-3319-4dc6-a9bd-6eb8342197d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:05.746355 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 721/1423] 10.222.0.15 () {58 vars in 1058 bytes} [Mon Oct 31 12:33:05 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:33:05.747337 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eff4aba-d584-4140-a936-1563c67769b8 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:05.751590 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 703/1424] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:33:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:33:05.754648 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f84017-ba21-49c1-a727-3ca67e140b7f None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:05.754818 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f84017-ba21-49c1-a727-3ca67e140b7f None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:05.755095 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f84017-ba21-49c1-a727-3ca67e140b7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:05.758496 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-0eff4aba-d584-4140-a936-1563c67769b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:05.796441 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-f9f84017-ba21-49c1-a727-3ca67e140b7f None None] MFA Rules not processed for user `db2e978bcb824a5e9d593b861c9e3bd7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:33:05.797677 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-f9f84017-ba21-49c1-a727-3ca67e140b7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:05.803333 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:33:05.880314 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 704/1425] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:33:05 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:33:05.881750 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763ef7e0-000f-4ad0-91e5-fd252d1aef8a None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:05.883694 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763ef7e0-000f-4ad0-91e5-fd252d1aef8a None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:05.883852 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763ef7e0-000f-4ad0-91e5-fd252d1aef8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:05.920539 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-763ef7e0-000f-4ad0-91e5-fd252d1aef8a None None] MFA Rules not processed for user `b2c03f0365ee4a26b8c076ae365325b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:33:05.921273 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-763ef7e0-000f-4ad0-91e5-fd252d1aef8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:05.938778 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eff4aba-d584-4140-a936-1563c67769b8 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:05.951203 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e94298-1d86-45cf-b077-127c00e3953c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99889b1dab0b471d9c77887cffefc652', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:05.951747 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e94298-1d86-45cf-b077-127c00e3953c service nova] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:05.953711 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e94298-1d86-45cf-b077-127c00e3953c service nova] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:05.954008 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e94298-1d86-45cf-b077-127c00e3953c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:05.959391 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-92e94298-1d86-45cf-b077-127c00e3953c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:06.060382 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e94298-1d86-45cf-b077-127c00e3953c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:06.062577 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e94298-1d86-45cf-b077-127c00e3953c service nova] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:06.074573 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 705/1426] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:33:05 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:33:06.080508 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763ef7e0-000f-4ad0-91e5-fd252d1aef8a None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:06.084704 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 722/1427] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:33:05 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:33:06.085577 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e94298-1d86-45cf-b077-127c00e3953c service nova] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:06.104232 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763ef7e0-000f-4ad0-91e5-fd252d1aef8a None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:06.105473 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e94298-1d86-45cf-b077-127c00e3953c service nova] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:06.114553 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d5fb71-e827-43f0-bd7b-49c1102ed663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:06.115568 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d5fb71-e827-43f0-bd7b-49c1102ed663 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:06.115849 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d5fb71-e827-43f0-bd7b-49c1102ed663 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:06.116292 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d5fb71-e827-43f0-bd7b-49c1102ed663 admin admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:06.116752 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d5fb71-e827-43f0-bd7b-49c1102ed663 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:33:06.117051 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d5fb71-e827-43f0-bd7b-49c1102ed663 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:06.118983 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97de4f08-fa69-4ddd-be45-9e88785a28f6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e7777495d7a406c8c50b1ad52e47263', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:06.119619 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97de4f08-fa69-4ddd-be45-9e88785a28f6 service placement] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:06.119891 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97de4f08-fa69-4ddd-be45-9e88785a28f6 service placement] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:06.120160 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97de4f08-fa69-4ddd-be45-9e88785a28f6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:06.122323 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d5fb71-e827-43f0-bd7b-49c1102ed663 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:06.137377 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97de4f08-fa69-4ddd-be45-9e88785a28f6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:06.141959 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97de4f08-fa69-4ddd-be45-9e88785a28f6 service placement] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:06.150882 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 723/1428] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Oct 31 12:33:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:33:06.152850 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d5fb71-e827-43f0-bd7b-49c1102ed663 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:06.159431 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-88d5fb71-e827-43f0-bd7b-49c1102ed663 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:06.184782 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 706/1429] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:33:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:33:06.186436 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97de4f08-fa69-4ddd-be45-9e88785a28f6 service placement] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:06.197342 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d5fb71-e827-43f0-bd7b-49c1102ed663 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:06.217301 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6d8bd8-574b-470b-aa19-66e2d720b782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:06.219005 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6d8bd8-574b-470b-aa19-66e2d720b782 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:06.219244 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97de4f08-fa69-4ddd-be45-9e88785a28f6 service placement] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:06.219446 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6d8bd8-574b-470b-aa19-66e2d720b782 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:06.219794 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6d8bd8-574b-470b-aa19-66e2d720b782 admin admin] PATH_INFO: `/v3/projects/8ae36f67568d4091b640ef57ce7e1e51` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:06.222620 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6d8bd8-574b-470b-aa19-66e2d720b782 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ae36f67568d4091b640ef57ce7e1e51)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:06.228487 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ede0b8-43bd-4791-8b62-b029fc2bec33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:06.229307 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ede0b8-43bd-4791-8b62-b029fc2bec33 admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:06.229550 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6d8bd8-574b-470b-aa19-66e2d720b782 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:06.229846 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ede0b8-43bd-4791-8b62-b029fc2bec33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:06.230140 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ede0b8-43bd-4791-8b62-b029fc2bec33 admin admin] PATH_INFO: `/v3/projects` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:06.230759 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ede0b8-43bd-4791-8b62-b029fc2bec33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:06.234694 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ede0b8-43bd-4791-8b62-b029fc2bec33 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:06.328350 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 707/1430] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:33:06 2022] POST /identity/v3/projects => generated 387 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:33:06.348995 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ede0b8-43bd-4791-8b62-b029fc2bec33 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:06.391596 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-ce6d8bd8-574b-470b-aa19-66e2d720b782 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ae36f67568d4091b640ef57ce7e1e51', 'request_id': 'req-ce6d8bd8-574b-470b-aa19-66e2d720b782'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:06.412825 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-ce6d8bd8-574b-470b-aa19-66e2d720b782 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ae36f67568d4091b640ef57ce7e1e51', 'request_id': 'req-ce6d8bd8-574b-470b-aa19-66e2d720b782'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:06.435469 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-ce6d8bd8-574b-470b-aa19-66e2d720b782 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ae36f67568d4091b640ef57ce7e1e51', 'request_id': 'req-ce6d8bd8-574b-470b-aa19-66e2d720b782'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:06.470848 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-ce6d8bd8-574b-470b-aa19-66e2d720b782 admin admin] The token cache is being invalidate because project 8ae36f67568d4091b640ef57ce7e1e51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:33:06.471587 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-ce6d8bd8-574b-470b-aa19-66e2d720b782 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:06.473157 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-ce6d8bd8-574b-470b-aa19-66e2d720b782 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:06.479689 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 724/1431] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:33:06 2022] DELETE /identity/v3/projects/8ae36f67568d4091b640ef57ce7e1e51 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:33:06.481204 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6d8bd8-574b-470b-aa19-66e2d720b782 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:06.484638 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-ce6d8bd8-574b-470b-aa19-66e2d720b782 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:06.574312 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ede0b8-43bd-4791-8b62-b029fc2bec33 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:06.584801 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:33:06.592434 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a595f741-7c08-48ea-8a2f-0e4ab6be0b94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:06.593078 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a595f741-7c08-48ea-8a2f-0e4ab6be0b94 admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:06.593489 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a595f741-7c08-48ea-8a2f-0e4ab6be0b94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:06.593674 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a595f741-7c08-48ea-8a2f-0e4ab6be0b94 admin admin] PATH_INFO: `/v3/users` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:06.594450 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a595f741-7c08-48ea-8a2f-0e4ab6be0b94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:06.596478 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a595f741-7c08-48ea-8a2f-0e4ab6be0b94 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:06.657286 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 708/1432] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:33:06 2022] POST /identity/v3/users => generated 294 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:33:06.658241 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a595f741-7c08-48ea-8a2f-0e4ab6be0b94 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:06.662188 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-a595f741-7c08-48ea-8a2f-0e4ab6be0b94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:06.664016 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6d8bd8-574b-470b-aa19-66e2d720b782 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:06.681168 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58610558-f076-4df3-9a5d-d7a36057e979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:06.681836 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58610558-f076-4df3-9a5d-d7a36057e979 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:06.682110 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58610558-f076-4df3-9a5d-d7a36057e979 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:06.682385 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58610558-f076-4df3-9a5d-d7a36057e979 admin admin] PATH_INFO: `/v3/users/50c789f1583f404d9175186457f14231` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:06.686328 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:33:06.688048 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58610558-f076-4df3-9a5d-d7a36057e979 admin admin] RBAC: Authorizing `identity:delete_user(user_id=50c789f1583f404d9175186457f14231)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:06.690201 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58610558-f076-4df3-9a5d-d7a36057e979 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:06.706849 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a595f741-7c08-48ea-8a2f-0e4ab6be0b94 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:06.726562 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea88e6f-e197-4c86-9a88-a3a2d96c3031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:06.727255 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea88e6f-e197-4c86-9a88-a3a2d96c3031 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:06.727534 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea88e6f-e197-4c86-9a88-a3a2d96c3031 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:06.727807 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea88e6f-e197-4c86-9a88-a3a2d96c3031 admin admin] PATH_INFO: `/v3/projects/cbf8f3193ba2448190df58e256e7e4f6` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:06.744967 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea88e6f-e197-4c86-9a88-a3a2d96c3031 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbf8f3193ba2448190df58e256e7e4f6)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:06.755689 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea88e6f-e197-4c86-9a88-a3a2d96c3031 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:06.842811 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-58610558-f076-4df3-9a5d-d7a36057e979 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50c789f1583f404d9175186457f14231'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:06.867697 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-58610558-f076-4df3-9a5d-d7a36057e979 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50c789f1583f404d9175186457f14231'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:06.871957 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-58610558-f076-4df3-9a5d-d7a36057e979 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50c789f1583f404d9175186457f14231'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:06.872866 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-58610558-f076-4df3-9a5d-d7a36057e979 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50c789f1583f404d9175186457f14231'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:06.915299 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-58610558-f076-4df3-9a5d-d7a36057e979 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50c789f1583f404d9175186457f14231'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:06.959333 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 725/1433] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:33:06 2022] DELETE /identity/v3/users/50c789f1583f404d9175186457f14231 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:33:06.959333 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58610558-f076-4df3-9a5d-d7a36057e979 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:06.962961 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-58610558-f076-4df3-9a5d-d7a36057e979 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:07.013046 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-fea88e6f-e197-4c86-9a88-a3a2d96c3031 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbf8f3193ba2448190df58e256e7e4f6', 'request_id': 'req-fea88e6f-e197-4c86-9a88-a3a2d96c3031'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:07.032152 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-fea88e6f-e197-4c86-9a88-a3a2d96c3031 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbf8f3193ba2448190df58e256e7e4f6', 'request_id': 'req-fea88e6f-e197-4c86-9a88-a3a2d96c3031'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:07.061911 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-fea88e6f-e197-4c86-9a88-a3a2d96c3031 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbf8f3193ba2448190df58e256e7e4f6', 'request_id': 'req-fea88e6f-e197-4c86-9a88-a3a2d96c3031'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:07.107446 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-fea88e6f-e197-4c86-9a88-a3a2d96c3031 admin admin] The token cache is being invalidate because project cbf8f3193ba2448190df58e256e7e4f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:33:07.107446 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-fea88e6f-e197-4c86-9a88-a3a2d96c3031 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:07.107880 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-fea88e6f-e197-4c86-9a88-a3a2d96c3031 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:07.110394 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 709/1434] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:33:06 2022] DELETE /identity/v3/projects/cbf8f3193ba2448190df58e256e7e4f6 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:33:07.123341 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea88e6f-e197-4c86-9a88-a3a2d96c3031 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:07.129799 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-fea88e6f-e197-4c86-9a88-a3a2d96c3031 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:07.230299 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58610558-f076-4df3-9a5d-d7a36057e979 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:07.343325 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea88e6f-e197-4c86-9a88-a3a2d96c3031 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:07.343810 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:33:07.365771 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea83b67-64c1-49dc-97bd-6effa4a6b6b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:07.366415 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea83b67-64c1-49dc-97bd-6effa4a6b6b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:07.366694 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea83b67-64c1-49dc-97bd-6effa4a6b6b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:07.366967 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea83b67-64c1-49dc-97bd-6effa4a6b6b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:07.367648 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea83b67-64c1-49dc-97bd-6effa4a6b6b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:07.369237 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea83b67-64c1-49dc-97bd-6effa4a6b6b0 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:07.397694 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1eca0b-0009-42a5-aeb1-95da4723c827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:07.398605 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1eca0b-0009-42a5-aeb1-95da4723c827 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:07.399453 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1eca0b-0009-42a5-aeb1-95da4723c827 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:07.399873 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1eca0b-0009-42a5-aeb1-95da4723c827 admin admin] PATH_INFO: `/v3/users/e4104a0759914252ba5d93394c38faf8` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:07.402749 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1eca0b-0009-42a5-aeb1-95da4723c827 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4104a0759914252ba5d93394c38faf8)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:07.403494 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 726/1435] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:33:06 2022] GET /identity/v3/roles => generated 1420 bytes in 446 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:33:07.404333 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea83b67-64c1-49dc-97bd-6effa4a6b6b0 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:07.406068 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-7ea83b67-64c1-49dc-97bd-6effa4a6b6b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:07.408983 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1eca0b-0009-42a5-aeb1-95da4723c827 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:07.429964 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea83b67-64c1-49dc-97bd-6effa4a6b6b0 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:07.445452 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1408d3-1309-491e-8d27-0261ad996f17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:07.446111 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1408d3-1309-491e-8d27-0261ad996f17 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:07.446508 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1408d3-1309-491e-8d27-0261ad996f17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:07.446649 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1408d3-1309-491e-8d27-0261ad996f17 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:07.447445 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1408d3-1309-491e-8d27-0261ad996f17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:07.449291 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1408d3-1309-491e-8d27-0261ad996f17 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:07.498226 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 727/1436] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:33:07 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:33:07.504938 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1408d3-1309-491e-8d27-0261ad996f17 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:07.540630 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1408d3-1309-491e-8d27-0261ad996f17 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:07.572901 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5992b27-4726-4851-82f6-62910bb4233c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:07.592326 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5992b27-4726-4851-82f6-62910bb4233c admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:07.592850 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5992b27-4726-4851-82f6-62910bb4233c admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:07.592952 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5992b27-4726-4851-82f6-62910bb4233c admin admin] PATH_INFO: `/v3/projects/3e2ef1ef425d4618987b6b9189e2fd0c/users/aa39ef99287845d0a06a8fc06774bf2b/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:07.635021 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-7e1eca0b-0009-42a5-aeb1-95da4723c827 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4104a0759914252ba5d93394c38faf8'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:07.668668 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-7e1eca0b-0009-42a5-aeb1-95da4723c827 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4104a0759914252ba5d93394c38faf8'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:07.670233 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-7e1eca0b-0009-42a5-aeb1-95da4723c827 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4104a0759914252ba5d93394c38faf8'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:07.680043 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5992b27-4726-4851-82f6-62910bb4233c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e2ef1ef425d4618987b6b9189e2fd0c, user_id=aa39ef99287845d0a06a8fc06774bf2b, role_id=ca1a5695f8bd499aa0b8cedaeecd2cc6)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:07.691354 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5992b27-4726-4851-82f6-62910bb4233c admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:07.712126 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-7e1eca0b-0009-42a5-aeb1-95da4723c827 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4104a0759914252ba5d93394c38faf8'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:07.733253 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-7e1eca0b-0009-42a5-aeb1-95da4723c827 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4104a0759914252ba5d93394c38faf8'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:07.747513 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 710/1437] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:33:07 2022] DELETE /identity/v3/users/e4104a0759914252ba5d93394c38faf8 => generated 0 bytes in 627 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:33:07.762785 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 728/1438] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:33:07 2022] PUT /identity/v3/projects/3e2ef1ef425d4618987b6b9189e2fd0c/users/aa39ef99287845d0a06a8fc06774bf2b/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 31 12:33:07.764441 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c355a9-6828-4f45-a2b1-6fd1a776a716 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:07.764866 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c355a9-6828-4f45-a2b1-6fd1a776a716 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:07.765198 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c355a9-6828-4f45-a2b1-6fd1a776a716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:07.779892 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:33:07.848659 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-c5c355a9-6828-4f45-a2b1-6fd1a776a716 None None] MFA Rules not processed for user `aa39ef99287845d0a06a8fc06774bf2b`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:33:07.867961 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-c5c355a9-6828-4f45-a2b1-6fd1a776a716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:08.139150 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 711/1439] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:33:07 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 373 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:33:08.140519 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5992b27-4726-4851-82f6-62910bb4233c admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:08.142491 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-b5992b27-4726-4851-82f6-62910bb4233c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:08.335311 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5992b27-4726-4851-82f6-62910bb4233c admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:08.563714 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816c6ba8-693c-4c08-a7e8-8352704da2ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:08.566966 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816c6ba8-693c-4c08-a7e8-8352704da2ad admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:08.567230 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816c6ba8-693c-4c08-a7e8-8352704da2ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:08.571700 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816c6ba8-693c-4c08-a7e8-8352704da2ad admin admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:08.575838 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816c6ba8-693c-4c08-a7e8-8352704da2ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:08.586787 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816c6ba8-693c-4c08-a7e8-8352704da2ad admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:08.671828 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 729/1440] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:33:08 2022] POST /identity/v3/projects => generated 387 bytes in 532 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:33:08.675867 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c355a9-6828-4f45-a2b1-6fd1a776a716 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:08.691502 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c355a9-6828-4f45-a2b1-6fd1a776a716 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:08.795962 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816c6ba8-693c-4c08-a7e8-8352704da2ad admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:08.799052 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-816c6ba8-693c-4c08-a7e8-8352704da2ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:08.911838 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6c9930-e338-4a08-b6de-cb251a45a64d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:08.912522 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6c9930-e338-4a08-b6de-cb251a45a64d admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:08.912740 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6c9930-e338-4a08-b6de-cb251a45a64d admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:08.913012 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6c9930-e338-4a08-b6de-cb251a45a64d admin admin] PATH_INFO: `/v3/users` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:08.921284 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6c9930-e338-4a08-b6de-cb251a45a64d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:08.929454 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6c9930-e338-4a08-b6de-cb251a45a64d admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:08.974784 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:33:08.986458 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 712/1441] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:33:08 2022] POST /identity/v3/users => generated 300 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 31 12:33:08.991018 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6c9930-e338-4a08-b6de-cb251a45a64d admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:08.999635 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816c6ba8-693c-4c08-a7e8-8352704da2ad admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:09.005843 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99638cd6-1f81-4550-b020-89ddb162bc4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:09.006508 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99638cd6-1f81-4550-b020-89ddb162bc4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:09.006806 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99638cd6-1f81-4550-b020-89ddb162bc4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:09.007667 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99638cd6-1f81-4550-b020-89ddb162bc4f admin admin] PATH_INFO: `/v3/projects/80329fb711ad4e4ba5547af8f6f945e1` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:09.008844 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99638cd6-1f81-4550-b020-89ddb162bc4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=80329fb711ad4e4ba5547af8f6f945e1)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:09.011095 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99638cd6-1f81-4550-b020-89ddb162bc4f admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:09.016637 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6c9930-e338-4a08-b6de-cb251a45a64d admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:09.025124 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905a81c6-e7ca-43c5-8862-4ba5f8924e6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:09.025752 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905a81c6-e7ca-43c5-8862-4ba5f8924e6e admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:09.026182 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905a81c6-e7ca-43c5-8862-4ba5f8924e6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:09.026470 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905a81c6-e7ca-43c5-8862-4ba5f8924e6e admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:09.030006 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905a81c6-e7ca-43c5-8862-4ba5f8924e6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:09.033833 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905a81c6-e7ca-43c5-8862-4ba5f8924e6e admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:09.063348 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 713/1442] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:33:08 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 31 12:33:09.065883 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905a81c6-e7ca-43c5-8862-4ba5f8924e6e admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:09.097163 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905a81c6-e7ca-43c5-8862-4ba5f8924e6e admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:09.109304 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-99638cd6-1f81-4550-b020-89ddb162bc4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80329fb711ad4e4ba5547af8f6f945e1', 'request_id': 'req-99638cd6-1f81-4550-b020-89ddb162bc4f'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:09.119853 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-99638cd6-1f81-4550-b020-89ddb162bc4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80329fb711ad4e4ba5547af8f6f945e1', 'request_id': 'req-99638cd6-1f81-4550-b020-89ddb162bc4f'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:09.146328 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-99638cd6-1f81-4550-b020-89ddb162bc4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80329fb711ad4e4ba5547af8f6f945e1', 'request_id': 'req-99638cd6-1f81-4550-b020-89ddb162bc4f'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:09.177226 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-99638cd6-1f81-4550-b020-89ddb162bc4f admin admin] The token cache is being invalidate because project 80329fb711ad4e4ba5547af8f6f945e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:33:09.177941 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-99638cd6-1f81-4550-b020-89ddb162bc4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:09.181142 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-99638cd6-1f81-4550-b020-89ddb162bc4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:09.187231 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 730/1443] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:33:08 2022] DELETE /identity/v3/projects/80329fb711ad4e4ba5547af8f6f945e1 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:33:09.192707 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9471f891-ed7c-4f00-a90d-d339d0a72913 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:09.195103 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9471f891-ed7c-4f00-a90d-d339d0a72913 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:09.195572 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9471f891-ed7c-4f00-a90d-d339d0a72913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:09.250016 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-9471f891-ed7c-4f00-a90d-d339d0a72913 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:33:09.251558 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-9471f891-ed7c-4f00-a90d-d339d0a72913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:09.408306 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b6a7ea-9421-4600-b445-0b9516ad2231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:09.410780 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b6a7ea-9421-4600-b445-0b9516ad2231 admin admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:09.411039 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b6a7ea-9421-4600-b445-0b9516ad2231 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:09.414455 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b6a7ea-9421-4600-b445-0b9516ad2231 admin admin] PATH_INFO: `/v3/projects/2441e30562e94e30bf3b9076ceed5f67/users/d3d906be03e34cb98bb5540cd91d17ef/roles/176c07b4fe3b4537bdfeb4be21b379c5` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:09.470472 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b6a7ea-9421-4600-b445-0b9516ad2231 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2441e30562e94e30bf3b9076ceed5f67, user_id=d3d906be03e34cb98bb5540cd91d17ef, role_id=176c07b4fe3b4537bdfeb4be21b379c5)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:09.474974 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b6a7ea-9421-4600-b445-0b9516ad2231 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:09.476745 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 731/1444] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:33:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 31 12:33:09.477722 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9471f891-ed7c-4f00-a90d-d339d0a72913 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:09.481820 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-9471f891-ed7c-4f00-a90d-d339d0a72913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:09.486976 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:33:09.519195 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 714/1445] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:33:09 2022] PUT /identity/v3/projects/2441e30562e94e30bf3b9076ceed5f67/users/d3d906be03e34cb98bb5540cd91d17ef/roles/176c07b4fe3b4537bdfeb4be21b379c5 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:33:09.522909 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9471f891-ed7c-4f00-a90d-d339d0a72913 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:09.524231 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324f9553-fca7-4cbc-9a7c-c47cf4474310 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:09.524609 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324f9553-fca7-4cbc-9a7c-c47cf4474310 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:09.524788 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324f9553-fca7-4cbc-9a7c-c47cf4474310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:09.564629 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-324f9553-fca7-4cbc-9a7c-c47cf4474310 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:33:09.568103 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-324f9553-fca7-4cbc-9a7c-c47cf4474310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:09.711500 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16abfad-33b4-4402-b633-e7fa395a3548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:09.713955 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16abfad-33b4-4402-b633-e7fa395a3548 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:09.714316 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16abfad-33b4-4402-b633-e7fa395a3548 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:09.714757 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16abfad-33b4-4402-b633-e7fa395a3548 admin admin] PATH_INFO: `/v3/projects/82311da4f89c402a854f67e5aade87da` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:09.716845 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16abfad-33b4-4402-b633-e7fa395a3548 admin admin] RBAC: Authorizing `identity:delete_project(project_id=82311da4f89c402a854f67e5aade87da)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:09.723627 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16abfad-33b4-4402-b633-e7fa395a3548 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:09.724819 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 715/1446] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:33:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:33:09.726928 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324f9553-fca7-4cbc-9a7c-c47cf4474310 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:09.729455 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-324f9553-fca7-4cbc-9a7c-c47cf4474310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:09.760972 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324f9553-fca7-4cbc-9a7c-c47cf4474310 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:09.782105 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a3f06c-9e31-4105-98cf-413473792289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:09.783370 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a3f06c-9e31-4105-98cf-413473792289 admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:09.783370 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a3f06c-9e31-4105-98cf-413473792289 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:09.783553 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a3f06c-9e31-4105-98cf-413473792289 admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:09.784880 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a3f06c-9e31-4105-98cf-413473792289 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:09.787643 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a3f06c-9e31-4105-98cf-413473792289 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:09.806342 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 716/1447] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:33:09 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:33:09.808226 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a3f06c-9e31-4105-98cf-413473792289 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:09.822936 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a3f06c-9e31-4105-98cf-413473792289 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:09.830381 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a313509-1fac-402b-9e5b-b050b73e9e1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:09.830720 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a313509-1fac-402b-9e5b-b050b73e9e1c admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:09.832675 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a313509-1fac-402b-9e5b-b050b73e9e1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:09.832955 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a313509-1fac-402b-9e5b-b050b73e9e1c admin admin] PATH_INFO: `/v3/domains` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:09.833636 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a313509-1fac-402b-9e5b-b050b73e9e1c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:33:09.834596 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a313509-1fac-402b-9e5b-b050b73e9e1c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:09.836057 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a313509-1fac-402b-9e5b-b050b73e9e1c admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:09.853729 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:33:09.863262 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 717/1448] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Oct 31 12:33:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:33:09.864190 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a313509-1fac-402b-9e5b-b050b73e9e1c admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:09.876690 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a313509-1fac-402b-9e5b-b050b73e9e1c admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:09.892733 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d891c90-0784-4655-89c9-d3a6b7d47549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:09.894805 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d891c90-0784-4655-89c9-d3a6b7d47549 admin admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:09.895096 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d891c90-0784-4655-89c9-d3a6b7d47549 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:09.895451 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d891c90-0784-4655-89c9-d3a6b7d47549 admin admin] PATH_INFO: `/v3/projects/2441e30562e94e30bf3b9076ceed5f67/users/d3d906be03e34cb98bb5540cd91d17ef/roles/176c07b4fe3b4537bdfeb4be21b379c5` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:09.900009 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d891c90-0784-4655-89c9-d3a6b7d47549 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2441e30562e94e30bf3b9076ceed5f67, user_id=d3d906be03e34cb98bb5540cd91d17ef, role_id=176c07b4fe3b4537bdfeb4be21b379c5)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:09.908146 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d891c90-0784-4655-89c9-d3a6b7d47549 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:09.937511 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 718/1449] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:33:09 2022] PUT /identity/v3/projects/2441e30562e94e30bf3b9076ceed5f67/users/d3d906be03e34cb98bb5540cd91d17ef/roles/176c07b4fe3b4537bdfeb4be21b379c5 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:33:09.939973 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d891c90-0784-4655-89c9-d3a6b7d47549 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:09.956384 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-b16abfad-33b4-4402-b633-e7fa395a3548 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82311da4f89c402a854f67e5aade87da', 'request_id': 'req-b16abfad-33b4-4402-b633-e7fa395a3548'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:09.974081 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-b16abfad-33b4-4402-b633-e7fa395a3548 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82311da4f89c402a854f67e5aade87da', 'request_id': 'req-b16abfad-33b4-4402-b633-e7fa395a3548'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:09.989154 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-b16abfad-33b4-4402-b633-e7fa395a3548 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82311da4f89c402a854f67e5aade87da', 'request_id': 'req-b16abfad-33b4-4402-b633-e7fa395a3548'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:10.067246 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-b16abfad-33b4-4402-b633-e7fa395a3548 admin admin] The token cache is being invalidate because project 82311da4f89c402a854f67e5aade87da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:33:10.067538 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-b16abfad-33b4-4402-b633-e7fa395a3548 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:10.072791 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d891c90-0784-4655-89c9-d3a6b7d47549 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:10.075651 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-b16abfad-33b4-4402-b633-e7fa395a3548 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:10.078617 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 732/1450] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:33:09 2022] DELETE /identity/v3/projects/82311da4f89c402a854f67e5aade87da => generated 0 bytes in 601 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:33:10.081942 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926edb15-e227-4d1b-84bc-726c0a7c7811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:10.081942 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926edb15-e227-4d1b-84bc-726c0a7c7811 admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:10.081942 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926edb15-e227-4d1b-84bc-726c0a7c7811 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:10.083049 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926edb15-e227-4d1b-84bc-726c0a7c7811 admin admin] PATH_INFO: `/v3/projects` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:10.083929 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926edb15-e227-4d1b-84bc-726c0a7c7811 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:10.084913 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1771ab2-2f41-41c6-a8f2-8f973b8be1d5 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:10.085359 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1771ab2-2f41-41c6-a8f2-8f973b8be1d5 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:10.085794 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1771ab2-2f41-41c6-a8f2-8f973b8be1d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:10.087411 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926edb15-e227-4d1b-84bc-726c0a7c7811 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:10.122042 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 719/1451] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:33:09 2022] POST /identity/v3/projects => generated 381 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:33:10.127126 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9946b9-5086-4aa5-81fa-e2e5100bf23f None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:10.127508 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9946b9-5086-4aa5-81fa-e2e5100bf23f None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:10.128892 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9946b9-5086-4aa5-81fa-e2e5100bf23f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:10.154716 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-ab9946b9-5086-4aa5-81fa-e2e5100bf23f None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:33:10.156474 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-ab9946b9-5086-4aa5-81fa-e2e5100bf23f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:10.166733 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-d1771ab2-2f41-41c6-a8f2-8f973b8be1d5 None None] MFA Rules not processed for user `d3d906be03e34cb98bb5540cd91d17ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:33:10.168281 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-d1771ab2-2f41-41c6-a8f2-8f973b8be1d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:10.228147 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:33:10.270179 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 720/1452] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:33:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:33:10.271646 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9946b9-5086-4aa5-81fa-e2e5100bf23f None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:10.273518 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-ab9946b9-5086-4aa5-81fa-e2e5100bf23f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:10.301964 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9946b9-5086-4aa5-81fa-e2e5100bf23f None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:10.310408 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 733/1453] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:33:10 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:33:10.312147 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bc1cfe-96de-4697-9e9e-ca933ed9f10c None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:10.312519 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bc1cfe-96de-4697-9e9e-ca933ed9f10c None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:10.312689 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bc1cfe-96de-4697-9e9e-ca933ed9f10c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:10.342774 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dc8eaf-8321-4a03-87e7-136e267b9b1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:10.343491 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc8eaf-8321-4a03-87e7-136e267b9b1d admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:10.343764 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc8eaf-8321-4a03-87e7-136e267b9b1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:10.344026 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc8eaf-8321-4a03-87e7-136e267b9b1d admin admin] PATH_INFO: `/v3/users` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:10.345769 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dc8eaf-8321-4a03-87e7-136e267b9b1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:10.347861 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dc8eaf-8321-4a03-87e7-136e267b9b1d admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:10.403805 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 721/1454] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:33:10 2022] POST /identity/v3/users => generated 291 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:33:10.403805 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada0851f-394b-47cd-8fb6-4db1cd07fcea None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:10.403805 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada0851f-394b-47cd-8fb6-4db1cd07fcea None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:10.403805 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada0851f-394b-47cd-8fb6-4db1cd07fcea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:10.459466 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-93bc1cfe-96de-4697-9e9e-ca933ed9f10c None None] MFA Rules not processed for user `bf4faacf7df248d28f32d7014dadcc07`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:33:10.461070 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-93bc1cfe-96de-4697-9e9e-ca933ed9f10c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:10.493209 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-ada0851f-394b-47cd-8fb6-4db1cd07fcea None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:33:10.494593 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-ada0851f-394b-47cd-8fb6-4db1cd07fcea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:10.511896 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 722/1455] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:33:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:33:10.513088 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada0851f-394b-47cd-8fb6-4db1cd07fcea None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:10.514704 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-ada0851f-394b-47cd-8fb6-4db1cd07fcea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:10.668438 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 734/1456] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Oct 31 12:33:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 357 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:33:10.670276 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bc1cfe-96de-4697-9e9e-ca933ed9f10c None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:10.686351 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:33:10.699155 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bc1cfe-96de-4697-9e9e-ca933ed9f10c None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:10.711612 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada0851f-394b-47cd-8fb6-4db1cd07fcea None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:10.715009 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065a2f88-0d6a-47e6-995b-3c2c95db64f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:10.717109 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065a2f88-0d6a-47e6-995b-3c2c95db64f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:10.717405 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065a2f88-0d6a-47e6-995b-3c2c95db64f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:10.718203 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065a2f88-0d6a-47e6-995b-3c2c95db64f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:10.720410 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065a2f88-0d6a-47e6-995b-3c2c95db64f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:10.725320 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065a2f88-0d6a-47e6-995b-3c2c95db64f4 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:10.728132 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cee007-223b-4faa-96e1-059f9ef126f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d6d7ea077d49e8a66c6c6762499702', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:10.728863 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cee007-223b-4faa-96e1-059f9ef126f0 service cinder] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:10.729143 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cee007-223b-4faa-96e1-059f9ef126f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:10.729412 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cee007-223b-4faa-96e1-059f9ef126f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:10.748359 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cee007-223b-4faa-96e1-059f9ef126f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:10.750862 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cee007-223b-4faa-96e1-059f9ef126f0 service cinder] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:10.756388 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 735/1457] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:33:10 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:33:10.757365 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065a2f88-0d6a-47e6-995b-3c2c95db64f4 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:10.775479 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 723/1458] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:33:10 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:33:10.776337 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cee007-223b-4faa-96e1-059f9ef126f0 service cinder] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:10.800536 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065a2f88-0d6a-47e6-995b-3c2c95db64f4 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:10.808263 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cee007-223b-4faa-96e1-059f9ef126f0 service cinder] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:10.814223 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fdb2a4-f4be-4ce0-b11f-e03c0c6e232a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:10.815794 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fdb2a4-f4be-4ce0-b11f-e03c0c6e232a admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:10.816134 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fdb2a4-f4be-4ce0-b11f-e03c0c6e232a admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:10.816407 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fdb2a4-f4be-4ce0-b11f-e03c0c6e232a admin admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:10.817188 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fdb2a4-f4be-4ce0-b11f-e03c0c6e232a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:33:10.817494 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fdb2a4-f4be-4ce0-b11f-e03c0c6e232a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:10.821028 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fdb2a4-f4be-4ce0-b11f-e03c0c6e232a admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:10.826945 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5abecc1-17ec-4711-adb6-c6259310008b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:10.828294 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5abecc1-17ec-4711-adb6-c6259310008b admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:10.828624 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5abecc1-17ec-4711-adb6-c6259310008b admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:10.829080 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5abecc1-17ec-4711-adb6-c6259310008b admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:10.829984 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5abecc1-17ec-4711-adb6-c6259310008b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:10.831780 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5abecc1-17ec-4711-adb6-c6259310008b admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:10.844854 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 724/1459] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:33:10 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:33:10.849557 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5abecc1-17ec-4711-adb6-c6259310008b admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:10.864323 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5abecc1-17ec-4711-adb6-c6259310008b admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:10.876893 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 736/1460] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Oct 31 12:33:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:33:10.877380 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc96d9f-512e-4b81-a999-b7b2be7f78c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:10.879790 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc96d9f-512e-4b81-a999-b7b2be7f78c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:10.880076 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc96d9f-512e-4b81-a999-b7b2be7f78c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:10.880352 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc96d9f-512e-4b81-a999-b7b2be7f78c5 admin admin] PATH_INFO: `/v3/projects/c3b3fabfa5054f74928712bae5020428/users/536efb4a565a4c2d938a75f212c3affc/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:10.882780 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fdb2a4-f4be-4ce0-b11f-e03c0c6e232a admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:10.912647 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fdb2a4-f4be-4ce0-b11f-e03c0c6e232a admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:10.939987 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961fd36d-b83c-45a0-9edc-1a8bfdd18dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:10.941322 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961fd36d-b83c-45a0-9edc-1a8bfdd18dbf admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:10.942182 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961fd36d-b83c-45a0-9edc-1a8bfdd18dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:10.942449 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961fd36d-b83c-45a0-9edc-1a8bfdd18dbf admin admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:10.943088 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961fd36d-b83c-45a0-9edc-1a8bfdd18dbf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:10.945034 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961fd36d-b83c-45a0-9edc-1a8bfdd18dbf admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:11.004912 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc96d9f-512e-4b81-a999-b7b2be7f78c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3b3fabfa5054f74928712bae5020428, user_id=536efb4a565a4c2d938a75f212c3affc, role_id=ca1a5695f8bd499aa0b8cedaeecd2cc6)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:11.036923 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc96d9f-512e-4b81-a999-b7b2be7f78c5 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:11.118240 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 725/1461] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:33:10 2022] PUT /identity/v3/projects/c3b3fabfa5054f74928712bae5020428/users/536efb4a565a4c2d938a75f212c3affc/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:33:11.118992 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc96d9f-512e-4b81-a999-b7b2be7f78c5 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:11.130572 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-5fc96d9f-512e-4b81-a999-b7b2be7f78c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:11.140244 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 737/1462] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:33:10 2022] POST /identity/v3/projects => generated 367 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:33:11.141115 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961fd36d-b83c-45a0-9edc-1a8bfdd18dbf admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:11.147128 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-961fd36d-b83c-45a0-9edc-1a8bfdd18dbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:11.329115 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc96d9f-512e-4b81-a999-b7b2be7f78c5 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:11.435276 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:33:11.437117 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961fd36d-b83c-45a0-9edc-1a8bfdd18dbf admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:11.457421 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73008e4e-adcf-41dd-bf99-7df1c3b1633b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2254a5786c249b4aa79244f7198ae65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:11.458061 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73008e4e-adcf-41dd-bf99-7df1c3b1633b service swift] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:11.458338 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73008e4e-adcf-41dd-bf99-7df1c3b1633b service swift] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:11.458604 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73008e4e-adcf-41dd-bf99-7df1c3b1633b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:11.510698 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72de9959-8a1f-483b-8f3a-f37af66cf8e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:11.511452 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72de9959-8a1f-483b-8f3a-f37af66cf8e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:11.511730 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72de9959-8a1f-483b-8f3a-f37af66cf8e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:11.511997 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72de9959-8a1f-483b-8f3a-f37af66cf8e3 admin admin] PATH_INFO: `/v3/users/aa39ef99287845d0a06a8fc06774bf2b` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:11.524120 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72de9959-8a1f-483b-8f3a-f37af66cf8e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa39ef99287845d0a06a8fc06774bf2b)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:11.526383 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72de9959-8a1f-483b-8f3a-f37af66cf8e3 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:11.627060 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73008e4e-adcf-41dd-bf99-7df1c3b1633b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:11.629905 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73008e4e-adcf-41dd-bf99-7df1c3b1633b service swift] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:11.641760 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 738/1463] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Oct 31 12:33:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 31 12:33:11.645052 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce11c5b-8cba-44a3-810e-0a158debeab6 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:11.645878 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce11c5b-8cba-44a3-810e-0a158debeab6 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:11.647136 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce11c5b-8cba-44a3-810e-0a158debeab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:11.660315 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-72de9959-8a1f-483b-8f3a-f37af66cf8e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa39ef99287845d0a06a8fc06774bf2b'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:11.673354 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-72de9959-8a1f-483b-8f3a-f37af66cf8e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa39ef99287845d0a06a8fc06774bf2b'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:11.673991 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-72de9959-8a1f-483b-8f3a-f37af66cf8e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa39ef99287845d0a06a8fc06774bf2b'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:11.689712 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-5ce11c5b-8cba-44a3-810e-0a158debeab6 None None] MFA Rules not processed for user `536efb4a565a4c2d938a75f212c3affc`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:33:11.693370 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-5ce11c5b-8cba-44a3-810e-0a158debeab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:11.698461 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-72de9959-8a1f-483b-8f3a-f37af66cf8e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa39ef99287845d0a06a8fc06774bf2b'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:11.726764 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-72de9959-8a1f-483b-8f3a-f37af66cf8e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa39ef99287845d0a06a8fc06774bf2b'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:11.760955 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 726/1464] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:33:11 2022] DELETE /identity/v3/users/aa39ef99287845d0a06a8fc06774bf2b => generated 0 bytes in 640 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:33:11.762379 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72de9959-8a1f-483b-8f3a-f37af66cf8e3 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:11.770206 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-72de9959-8a1f-483b-8f3a-f37af66cf8e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:11.842717 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:33:11.975217 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 739/1465] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:33:11 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 332 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:33:11.976529 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93ebd28-aa21-49b4-b3c5-b3e53b462f4d None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:11.976810 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93ebd28-aa21-49b4-b3c5-b3e53b462f4d None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:11.977079 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93ebd28-aa21-49b4-b3c5-b3e53b462f4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:12.056615 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72de9959-8a1f-483b-8f3a-f37af66cf8e3 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:12.075249 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-f93ebd28-aa21-49b4-b3c5-b3e53b462f4d None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:33:12.076940 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-f93ebd28-aa21-49b4-b3c5-b3e53b462f4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:12.079773 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2c2d2a-282a-43de-bc3c-ce3a79bb59de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:12.080466 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2c2d2a-282a-43de-bc3c-ce3a79bb59de admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:12.080712 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2c2d2a-282a-43de-bc3c-ce3a79bb59de admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:12.081197 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2c2d2a-282a-43de-bc3c-ce3a79bb59de admin admin] PATH_INFO: `/v3/users` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:12.082008 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2c2d2a-282a-43de-bc3c-ce3a79bb59de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:12.083980 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2c2d2a-282a-43de-bc3c-ce3a79bb59de admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:12.111955 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 740/1466] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:33:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:33:12.113591 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93ebd28-aa21-49b4-b3c5-b3e53b462f4d None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:12.116181 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-f93ebd28-aa21-49b4-b3c5-b3e53b462f4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:12.143101 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93ebd28-aa21-49b4-b3c5-b3e53b462f4d None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:12.151619 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 727/1467] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:33:11 2022] POST /identity/v3/users => generated 284 bytes in 393 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:33:12.155792 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2c2d2a-282a-43de-bc3c-ce3a79bb59de admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:12.159413 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2077c000-58ab-4ec4-99d0-3e190f794d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:12.160313 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2077c000-58ab-4ec4-99d0-3e190f794d14 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:12.164992 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2077c000-58ab-4ec4-99d0-3e190f794d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:12.165376 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2077c000-58ab-4ec4-99d0-3e190f794d14 admin admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:12.169388 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2077c000-58ab-4ec4-99d0-3e190f794d14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:12.171791 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2c2d2a-282a-43de-bc3c-ce3a79bb59de admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:12.173523 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2077c000-58ab-4ec4-99d0-3e190f794d14 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:12.180861 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06577f3-86e8-4c01-a50d-b2a57e536ad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:12.183581 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06577f3-86e8-4c01-a50d-b2a57e536ad9 admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:12.184189 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06577f3-86e8-4c01-a50d-b2a57e536ad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:12.186045 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06577f3-86e8-4c01-a50d-b2a57e536ad9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:12.186718 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06577f3-86e8-4c01-a50d-b2a57e536ad9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:12.192754 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06577f3-86e8-4c01-a50d-b2a57e536ad9 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:12.232906 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 728/1468] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:33:12 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 31 12:33:12.234797 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 741/1469] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:33:12 2022] POST /identity/v3/projects => generated 383 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:33:12.236225 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06577f3-86e8-4c01-a50d-b2a57e536ad9 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:12.241415 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-e06577f3-86e8-4c01-a50d-b2a57e536ad9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:12.244654 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2077c000-58ab-4ec4-99d0-3e190f794d14 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:12.358669 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:33:12.447376 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06577f3-86e8-4c01-a50d-b2a57e536ad9 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:12.510067 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2077c000-58ab-4ec4-99d0-3e190f794d14 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:12.524584 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d53771c-34c0-42b0-a8e5-8a251b3ef290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:12.525243 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d53771c-34c0-42b0-a8e5-8a251b3ef290 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:12.525520 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d53771c-34c0-42b0-a8e5-8a251b3ef290 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:12.525792 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d53771c-34c0-42b0-a8e5-8a251b3ef290 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:12.526450 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d53771c-34c0-42b0-a8e5-8a251b3ef290 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:12.535340 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d53771c-34c0-42b0-a8e5-8a251b3ef290 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:12.579130 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 742/1470] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:33:12 2022] GET /identity/v3/roles => generated 1420 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:33:12.580128 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d53771c-34c0-42b0-a8e5-8a251b3ef290 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:12.616767 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d53771c-34c0-42b0-a8e5-8a251b3ef290 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:12.633687 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930bc296-4fe6-42a1-b093-bf3a1720008c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:12.637130 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930bc296-4fe6-42a1-b093-bf3a1720008c admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:12.637482 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930bc296-4fe6-42a1-b093-bf3a1720008c admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:12.637811 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930bc296-4fe6-42a1-b093-bf3a1720008c admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:12.638493 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930bc296-4fe6-42a1-b093-bf3a1720008c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:12.640493 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930bc296-4fe6-42a1-b093-bf3a1720008c admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:12.667730 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2762b1-3779-4976-a404-0d1f8b0599d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:12.668666 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2762b1-3779-4976-a404-0d1f8b0599d5 service neutron] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:12.669428 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2762b1-3779-4976-a404-0d1f8b0599d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:12.669856 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2762b1-3779-4976-a404-0d1f8b0599d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:12.693834 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2762b1-3779-4976-a404-0d1f8b0599d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:12.699883 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2762b1-3779-4976-a404-0d1f8b0599d5 service neutron] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:12.722925 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 743/1471] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:33:12 2022] POST /identity/v3/users => generated 298 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:33:12.724521 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930bc296-4fe6-42a1-b093-bf3a1720008c admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:12.731594 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 729/1472] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:33:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:33:12.734476 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61c7ac7-c86a-433d-8e6d-0cc5340fda9b None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:12.735832 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61c7ac7-c86a-433d-8e6d-0cc5340fda9b None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:12.736211 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61c7ac7-c86a-433d-8e6d-0cc5340fda9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:12.754806 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930bc296-4fe6-42a1-b093-bf3a1720008c admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:12.764693 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5bbea3-0087-4dca-b04d-fd56567c6bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:12.765336 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5bbea3-0087-4dca-b04d-fd56567c6bc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:12.765613 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5bbea3-0087-4dca-b04d-fd56567c6bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:12.765879 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5bbea3-0087-4dca-b04d-fd56567c6bc8 admin admin] PATH_INFO: `/v3/projects/62d591a4cc824393886ced9e459a8002/users/8efc04798db9441cb1c461500b3fb03d/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:12.794895 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-b61c7ac7-c86a-433d-8e6d-0cc5340fda9b None None] MFA Rules not processed for user `a6d1ace68aa84499a4929da60f3d71bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:33:12.798521 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-b61c7ac7-c86a-433d-8e6d-0cc5340fda9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:12.801667 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5bbea3-0087-4dca-b04d-fd56567c6bc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62d591a4cc824393886ced9e459a8002, user_id=8efc04798db9441cb1c461500b3fb03d, role_id=ca1a5695f8bd499aa0b8cedaeecd2cc6)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:12.808639 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5bbea3-0087-4dca-b04d-fd56567c6bc8 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:12.848719 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 744/1473] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:33:12 2022] PUT /identity/v3/projects/62d591a4cc824393886ced9e459a8002/users/8efc04798db9441cb1c461500b3fb03d/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:33:12.850029 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5bbea3-0087-4dca-b04d-fd56567c6bc8 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:13.034661 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:33:13.040244 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 730/1474] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:33:12 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:33:13.044326 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6182c3-a4e3-4711-8efe-9b8dd1360fdc None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:13.044617 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6182c3-a4e3-4711-8efe-9b8dd1360fdc None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:13.048446 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5bbea3-0087-4dca-b04d-fd56567c6bc8 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:13.058075 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7917634f-50f5-45be-9f8c-8f53969c64bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:13.058719 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7917634f-50f5-45be-9f8c-8f53969c64bb admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:13.058997 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7917634f-50f5-45be-9f8c-8f53969c64bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:13.059315 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7917634f-50f5-45be-9f8c-8f53969c64bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:13.060169 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7917634f-50f5-45be-9f8c-8f53969c64bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:13.062881 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7917634f-50f5-45be-9f8c-8f53969c64bb admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:13.083325 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6182c3-a4e3-4711-8efe-9b8dd1360fdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:13.130912 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 745/1475] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:33:12 2022] GET /identity/v3/roles => generated 1420 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:33:13.138300 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7917634f-50f5-45be-9f8c-8f53969c64bb admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:13.223503 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7917634f-50f5-45be-9f8c-8f53969c64bb admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:13.245565 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-3d6182c3-a4e3-4711-8efe-9b8dd1360fdc None None] MFA Rules not processed for user `8efc04798db9441cb1c461500b3fb03d`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:33:13.247994 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-3d6182c3-a4e3-4711-8efe-9b8dd1360fdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:13.275749 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2175922-a9a9-4aba-9157-8cc046d94de1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:13.279522 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2175922-a9a9-4aba-9157-8cc046d94de1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:13.285268 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2175922-a9a9-4aba-9157-8cc046d94de1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:13.285729 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2175922-a9a9-4aba-9157-8cc046d94de1 admin admin] PATH_INFO: `/v3/projects/d549fbb3c6bf4014b6fc54ab5b45386d/users/d610922674754b598ace753064c1fb8f/roles/176c07b4fe3b4537bdfeb4be21b379c5` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:13.393050 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2175922-a9a9-4aba-9157-8cc046d94de1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d549fbb3c6bf4014b6fc54ab5b45386d, user_id=d610922674754b598ace753064c1fb8f, role_id=176c07b4fe3b4537bdfeb4be21b379c5)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:13.398203 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2175922-a9a9-4aba-9157-8cc046d94de1 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:13.459459 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 746/1476] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:33:13 2022] PUT /identity/v3/projects/d549fbb3c6bf4014b6fc54ab5b45386d/users/d610922674754b598ace753064c1fb8f/roles/176c07b4fe3b4537bdfeb4be21b379c5 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:33:13.464217 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2175922-a9a9-4aba-9157-8cc046d94de1 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:13.618673 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 731/1477] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:33:13 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 578 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:33:13.623108 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6182c3-a4e3-4711-8efe-9b8dd1360fdc None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:13.657284 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6182c3-a4e3-4711-8efe-9b8dd1360fdc None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:13.704615 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2175922-a9a9-4aba-9157-8cc046d94de1 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:13.724879 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27943a0e-dfdd-466b-a19e-9e87159ae77f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:13.727523 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27943a0e-dfdd-466b-a19e-9e87159ae77f admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:13.728015 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27943a0e-dfdd-466b-a19e-9e87159ae77f admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:13.728511 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27943a0e-dfdd-466b-a19e-9e87159ae77f admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:13.730765 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27943a0e-dfdd-466b-a19e-9e87159ae77f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:13.733640 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27943a0e-dfdd-466b-a19e-9e87159ae77f admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:13.779938 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:33:13.790880 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 747/1478] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:33:13 2022] GET /identity/v3/roles => generated 1420 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:33:13.793722 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27943a0e-dfdd-466b-a19e-9e87159ae77f admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:13.820291 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27943a0e-dfdd-466b-a19e-9e87159ae77f admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:13.835176 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c55d61-d7cd-49e4-bb64-1246eeedceab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:13.837681 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c55d61-d7cd-49e4-bb64-1246eeedceab admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:13.837982 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c55d61-d7cd-49e4-bb64-1246eeedceab admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:13.838864 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c55d61-d7cd-49e4-bb64-1246eeedceab admin admin] PATH_INFO: `/v3/projects/d549fbb3c6bf4014b6fc54ab5b45386d/users/d610922674754b598ace753064c1fb8f/roles/176c07b4fe3b4537bdfeb4be21b379c5` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:13.844522 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c55d61-d7cd-49e4-bb64-1246eeedceab admin admin] RBAC: Authorizing `identity:create_grant(project_id=d549fbb3c6bf4014b6fc54ab5b45386d, user_id=d610922674754b598ace753064c1fb8f, role_id=176c07b4fe3b4537bdfeb4be21b379c5)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:13.856086 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28f5f39-e6a2-4291-a5fd-ef2561a06006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:13.856728 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28f5f39-e6a2-4291-a5fd-ef2561a06006 admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:13.857006 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28f5f39-e6a2-4291-a5fd-ef2561a06006 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:13.857274 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28f5f39-e6a2-4291-a5fd-ef2561a06006 admin admin] PATH_INFO: `/v3/projects` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:13.860750 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28f5f39-e6a2-4291-a5fd-ef2561a06006 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:13.864287 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28f5f39-e6a2-4291-a5fd-ef2561a06006 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:13.864417 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c55d61-d7cd-49e4-bb64-1246eeedceab admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:13.898348 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 748/1479] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:33:13 2022] PUT /identity/v3/projects/d549fbb3c6bf4014b6fc54ab5b45386d/users/d610922674754b598ace753064c1fb8f/roles/176c07b4fe3b4537bdfeb4be21b379c5 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:33:13.914265 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b61fb49-a47c-4253-bb42-e03c8628be04 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:13.914566 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b61fb49-a47c-4253-bb42-e03c8628be04 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:13.915673 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b61fb49-a47c-4253-bb42-e03c8628be04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:13.972466 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 732/1480] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:33:13 2022] POST /identity/v3/projects => generated 365 bytes in 347 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 31 12:33:13.972466 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8c8ae9-8879-4df8-974c-f57a13b2ca92 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:13.972466 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8c8ae9-8879-4df8-974c-f57a13b2ca92 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:13.972466 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8c8ae9-8879-4df8-974c-f57a13b2ca92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:14.000082 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-4b61fb49-a47c-4253-bb42-e03c8628be04 None None] MFA Rules not processed for user `d610922674754b598ace753064c1fb8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:33:14.002065 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-4b61fb49-a47c-4253-bb42-e03c8628be04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:14.049871 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-df8c8ae9-8879-4df8-974c-f57a13b2ca92 None None] MFA Rules not processed for user `25e9c7d1e4714bf18a1ced84855423d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:33:14.051174 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-df8c8ae9-8879-4df8-974c-f57a13b2ca92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:14.208861 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 733/1481] 10.222.0.20 () {58 vars in 1015 bytes} [Mon Oct 31 12:33:13 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:33:14.209822 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8c8ae9-8879-4df8-974c-f57a13b2ca92 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:14.258417 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 749/1482] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:33:13 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 342 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:33:14.272683 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b61fb49-a47c-4253-bb42-e03c8628be04 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:14.275509 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-4b61fb49-a47c-4253-bb42-e03c8628be04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:14.486982 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8c8ae9-8879-4df8-974c-f57a13b2ca92 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:14.509163 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d6b4f2-e607-41a6-a672-bc01285b1ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:14.509805 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d6b4f2-e607-41a6-a672-bc01285b1ec7 admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:14.514141 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d6b4f2-e607-41a6-a672-bc01285b1ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:14.514520 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d6b4f2-e607-41a6-a672-bc01285b1ec7 admin admin] PATH_INFO: `/v3/users` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:14.515495 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d6b4f2-e607-41a6-a672-bc01285b1ec7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:14.531246 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d6b4f2-e607-41a6-a672-bc01285b1ec7 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:14.634938 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 734/1483] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:33:14 2022] POST /identity/v3/users => generated 289 bytes in 423 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:33:14.637008 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d6b4f2-e607-41a6-a672-bc01285b1ec7 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:14.672101 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d6b4f2-e607-41a6-a672-bc01285b1ec7 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:14.686610 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9572e419-fbfc-471c-aaee-1fbb0f5e4367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:14.687346 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9572e419-fbfc-471c-aaee-1fbb0f5e4367 admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:14.687700 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9572e419-fbfc-471c-aaee-1fbb0f5e4367 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:14.687977 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9572e419-fbfc-471c-aaee-1fbb0f5e4367 admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:14.688669 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9572e419-fbfc-471c-aaee-1fbb0f5e4367 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:14.695971 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9572e419-fbfc-471c-aaee-1fbb0f5e4367 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:14.721075 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 735/1484] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:33:14 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:33:14.722243 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9572e419-fbfc-471c-aaee-1fbb0f5e4367 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:14.736415 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9572e419-fbfc-471c-aaee-1fbb0f5e4367 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:14.743286 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e57e70-ff99-41a3-8aee-118db3008e59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:14.744332 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e57e70-ff99-41a3-8aee-118db3008e59 service neutron] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:14.744755 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e57e70-ff99-41a3-8aee-118db3008e59 service neutron] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:14.745467 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e57e70-ff99-41a3-8aee-118db3008e59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:14.779958 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e57e70-ff99-41a3-8aee-118db3008e59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:14.782833 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e57e70-ff99-41a3-8aee-118db3008e59 service neutron] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:14.805547 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 736/1485] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:33:14 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:33:14.807947 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e57e70-ff99-41a3-8aee-118db3008e59 service neutron] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:14.844552 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e57e70-ff99-41a3-8aee-118db3008e59 service neutron] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:14.853402 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dd4a8a-a2a7-4b0d-b5cc-429b3483f877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:14.854081 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dd4a8a-a2a7-4b0d-b5cc-429b3483f877 admin admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:14.854358 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dd4a8a-a2a7-4b0d-b5cc-429b3483f877 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:14.854624 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dd4a8a-a2a7-4b0d-b5cc-429b3483f877 admin admin] PATH_INFO: `/v3/projects/685a10b3658849e2b21e61ca1222bda6/users/0b8d3d4975c145e1adb7886c0539f918/roles/176c07b4fe3b4537bdfeb4be21b379c5` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:14.883535 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:33:14.919324 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dd4a8a-a2a7-4b0d-b5cc-429b3483f877 admin admin] RBAC: Authorizing `identity:create_grant(project_id=685a10b3658849e2b21e61ca1222bda6, user_id=0b8d3d4975c145e1adb7886c0539f918, role_id=176c07b4fe3b4537bdfeb4be21b379c5)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:14.926240 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dd4a8a-a2a7-4b0d-b5cc-429b3483f877 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:14.948415 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b61fb49-a47c-4253-bb42-e03c8628be04 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:14.955996 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 737/1486] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:33:14 2022] PUT /identity/v3/projects/685a10b3658849e2b21e61ca1222bda6/users/0b8d3d4975c145e1adb7886c0539f918/roles/176c07b4fe3b4537bdfeb4be21b379c5 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:33:14.962627 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dd4a8a-a2a7-4b0d-b5cc-429b3483f877 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:14.968765 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:33:15.147752 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dd4a8a-a2a7-4b0d-b5cc-429b3483f877 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:15.154218 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2d6119-32bc-4781-ae90-4794be77e2f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:15.161156 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2d6119-32bc-4781-ae90-4794be77e2f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:15.161682 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2d6119-32bc-4781-ae90-4794be77e2f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:15.162104 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2d6119-32bc-4781-ae90-4794be77e2f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:15.162923 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2d6119-32bc-4781-ae90-4794be77e2f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:15.166709 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2d6119-32bc-4781-ae90-4794be77e2f5 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:15.169169 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4303d623-8e03-434b-b85b-bbd545690f9d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d6d7ea077d49e8a66c6c6762499702', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:15.170559 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4303d623-8e03-434b-b85b-bbd545690f9d service cinder] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:15.170847 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4303d623-8e03-434b-b85b-bbd545690f9d service cinder] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:15.171303 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4303d623-8e03-434b-b85b-bbd545690f9d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:15.188135 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 738/1487] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:33:14 2022] GET /identity/v3/roles => generated 1420 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:33:15.192609 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2d6119-32bc-4781-ae90-4794be77e2f5 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:15.223665 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2d6119-32bc-4781-ae90-4794be77e2f5 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:15.247915 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4afc14b-3d90-46a2-ae48-90d396899b9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:15.249964 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4afc14b-3d90-46a2-ae48-90d396899b9c admin admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:15.251611 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4afc14b-3d90-46a2-ae48-90d396899b9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:15.253906 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4afc14b-3d90-46a2-ae48-90d396899b9c admin admin] PATH_INFO: `/v3/projects/685a10b3658849e2b21e61ca1222bda6/users/0b8d3d4975c145e1adb7886c0539f918/roles/176c07b4fe3b4537bdfeb4be21b379c5` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:15.259808 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4afc14b-3d90-46a2-ae48-90d396899b9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=685a10b3658849e2b21e61ca1222bda6, user_id=0b8d3d4975c145e1adb7886c0539f918, role_id=176c07b4fe3b4537bdfeb4be21b379c5)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:15.271402 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4afc14b-3d90-46a2-ae48-90d396899b9c admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:15.307803 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 739/1488] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:33:15 2022] PUT /identity/v3/projects/685a10b3658849e2b21e61ca1222bda6/users/0b8d3d4975c145e1adb7886c0539f918/roles/176c07b4fe3b4537bdfeb4be21b379c5 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:33:15.320291 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7eb3d2-5f97-422d-8c57-25cc9c817b2a None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:15.320898 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7eb3d2-5f97-422d-8c57-25cc9c817b2a None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:15.321435 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7eb3d2-5f97-422d-8c57-25cc9c817b2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:15.366458 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4303d623-8e03-434b-b85b-bbd545690f9d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:15.368988 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4303d623-8e03-434b-b85b-bbd545690f9d service cinder] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:15.396348 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 750/1489] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:33:14 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 1126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:33:15.453392 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-1c7eb3d2-5f97-422d-8c57-25cc9c817b2a None None] MFA Rules not processed for user `0b8d3d4975c145e1adb7886c0539f918`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:33:15.454731 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-1c7eb3d2-5f97-422d-8c57-25cc9c817b2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:15.684700 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4303d623-8e03-434b-b85b-bbd545690f9d service cinder] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:15.698288 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4303d623-8e03-434b-b85b-bbd545690f9d service cinder] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:15.758346 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 740/1490] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:33:15 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 438 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:33:15.759441 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7eb3d2-5f97-422d-8c57-25cc9c817b2a None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:15.765869 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-1c7eb3d2-5f97-422d-8c57-25cc9c817b2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:15.922611 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed350b57-2b5b-49f8-a22b-9fcd8ab40044 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d6d7ea077d49e8a66c6c6762499702', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:15.925907 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed350b57-2b5b-49f8-a22b-9fcd8ab40044 service cinder] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:15.934682 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed350b57-2b5b-49f8-a22b-9fcd8ab40044 service cinder] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:15.936242 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed350b57-2b5b-49f8-a22b-9fcd8ab40044 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:16.012330 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7eb3d2-5f97-422d-8c57-25cc9c817b2a None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:16.035480 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918931ca-d0a3-493a-b994-4e63e14444c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d6d7ea077d49e8a66c6c6762499702', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:16.036138 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918931ca-d0a3-493a-b994-4e63e14444c0 service cinder] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:16.036419 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918931ca-d0a3-493a-b994-4e63e14444c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:16.038772 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918931ca-d0a3-493a-b994-4e63e14444c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:16.071920 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918931ca-d0a3-493a-b994-4e63e14444c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:16.073352 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918931ca-d0a3-493a-b994-4e63e14444c0 service cinder] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:16.126933 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 741/1491] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:33:15 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:33:16.154913 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:33:16.167961 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed350b57-2b5b-49f8-a22b-9fcd8ab40044 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:16.172521 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed350b57-2b5b-49f8-a22b-9fcd8ab40044 service cinder] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:16.281469 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 751/1492] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:33:15 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 596 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 31 12:33:16.373712 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918931ca-d0a3-493a-b994-4e63e14444c0 service cinder] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:16.386285 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918931ca-d0a3-493a-b994-4e63e14444c0 service cinder] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:16.398656 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eccbb16-3a1f-48d7-b8d5-6aee4b185047 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d6d7ea077d49e8a66c6c6762499702', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:16.399359 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eccbb16-3a1f-48d7-b8d5-6aee4b185047 service cinder] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:16.399642 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eccbb16-3a1f-48d7-b8d5-6aee4b185047 service cinder] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:16.400086 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eccbb16-3a1f-48d7-b8d5-6aee4b185047 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:16.425397 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:33:16.555310 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eccbb16-3a1f-48d7-b8d5-6aee4b185047 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:16.556770 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eccbb16-3a1f-48d7-b8d5-6aee4b185047 service cinder] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:16.583360 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 742/1493] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:33:16 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:33:17.043415 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed350b57-2b5b-49f8-a22b-9fcd8ab40044 service cinder] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:17.045217 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-ed350b57-2b5b-49f8-a22b-9fcd8ab40044 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:17.093354 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eccbb16-3a1f-48d7-b8d5-6aee4b185047 service cinder] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:17.375837 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eccbb16-3a1f-48d7-b8d5-6aee4b185047 service cinder] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:17.408651 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c91d75-62cf-476e-95c0-d98b5aa1b42d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:17.410120 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c91d75-62cf-476e-95c0-d98b5aa1b42d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:17.410410 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c91d75-62cf-476e-95c0-d98b5aa1b42d admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:17.410687 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c91d75-62cf-476e-95c0-d98b5aa1b42d admin admin] PATH_INFO: `/v3/users/536efb4a565a4c2d938a75f212c3affc` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:17.416555 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c91d75-62cf-476e-95c0-d98b5aa1b42d admin admin] RBAC: Authorizing `identity:delete_user(user_id=536efb4a565a4c2d938a75f212c3affc)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:17.422462 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c91d75-62cf-476e-95c0-d98b5aa1b42d admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:17.425423 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed350b57-2b5b-49f8-a22b-9fcd8ab40044 service cinder] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:17.446240 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccaab88-480c-43e8-89bd-e4cce898ea71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:17.446926 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccaab88-480c-43e8-89bd-e4cce898ea71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:17.447307 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccaab88-480c-43e8-89bd-e4cce898ea71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:17.447516 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccaab88-480c-43e8-89bd-e4cce898ea71 admin admin] PATH_INFO: `/v3/projects/3e2ef1ef425d4618987b6b9189e2fd0c` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:17.461919 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccaab88-480c-43e8-89bd-e4cce898ea71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e2ef1ef425d4618987b6b9189e2fd0c)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:17.469100 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccaab88-480c-43e8-89bd-e4cce898ea71 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:17.595774 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-06c91d75-62cf-476e-95c0-d98b5aa1b42d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '536efb4a565a4c2d938a75f212c3affc'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:17.621224 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-06c91d75-62cf-476e-95c0-d98b5aa1b42d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '536efb4a565a4c2d938a75f212c3affc'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:17.621666 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-06c91d75-62cf-476e-95c0-d98b5aa1b42d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '536efb4a565a4c2d938a75f212c3affc'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:17.672394 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-06c91d75-62cf-476e-95c0-d98b5aa1b42d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '536efb4a565a4c2d938a75f212c3affc'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:17.698653 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-06c91d75-62cf-476e-95c0-d98b5aa1b42d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '536efb4a565a4c2d938a75f212c3affc'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:17.708010 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-2ccaab88-480c-43e8-89bd-e4cce898ea71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e2ef1ef425d4618987b6b9189e2fd0c', 'request_id': 'req-2ccaab88-480c-43e8-89bd-e4cce898ea71'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:17.723094 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 743/1494] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:33:17 2022] DELETE /identity/v3/users/536efb4a565a4c2d938a75f212c3affc => generated 0 bytes in 636 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 31 12:33:17.727943 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608009cf-3ca9-4acd-b723-b6a2eae48c2d None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:17.728236 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608009cf-3ca9-4acd-b723-b6a2eae48c2d None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:17.728513 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608009cf-3ca9-4acd-b723-b6a2eae48c2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:17.735941 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-2ccaab88-480c-43e8-89bd-e4cce898ea71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e2ef1ef425d4618987b6b9189e2fd0c', 'request_id': 'req-2ccaab88-480c-43e8-89bd-e4cce898ea71'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:17.753464 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-2ccaab88-480c-43e8-89bd-e4cce898ea71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e2ef1ef425d4618987b6b9189e2fd0c', 'request_id': 'req-2ccaab88-480c-43e8-89bd-e4cce898ea71'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:17.777744 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-2ccaab88-480c-43e8-89bd-e4cce898ea71 admin admin] The token cache is being invalidate because project 3e2ef1ef425d4618987b6b9189e2fd0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:33:17.777994 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-2ccaab88-480c-43e8-89bd-e4cce898ea71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:17.798420 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-2ccaab88-480c-43e8-89bd-e4cce898ea71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:17.800451 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 752/1495] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:33:17 2022] DELETE /identity/v3/projects/3e2ef1ef425d4618987b6b9189e2fd0c => generated 0 bytes in 758 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:33:17.804642 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccaab88-480c-43e8-89bd-e4cce898ea71 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:17.806063 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-2ccaab88-480c-43e8-89bd-e4cce898ea71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:17.810144 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-608009cf-3ca9-4acd-b723-b6a2eae48c2d None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:33:17.811842 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-608009cf-3ca9-4acd-b723-b6a2eae48c2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:17.929286 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:33:18.067587 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 744/1496] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:33:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:33:18.068277 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccaab88-480c-43e8-89bd-e4cce898ea71 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:18.084614 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94cc76b-a79d-4bf1-8c7f-2995aaf1ffb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:18.085284 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94cc76b-a79d-4bf1-8c7f-2995aaf1ffb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:18.085564 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94cc76b-a79d-4bf1-8c7f-2995aaf1ffb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:18.085840 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94cc76b-a79d-4bf1-8c7f-2995aaf1ffb4 admin admin] PATH_INFO: `/v3/users/d3d906be03e34cb98bb5540cd91d17ef` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:18.088762 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94cc76b-a79d-4bf1-8c7f-2995aaf1ffb4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3d906be03e34cb98bb5540cd91d17ef)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:18.091089 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94cc76b-a79d-4bf1-8c7f-2995aaf1ffb4 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:18.125157 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608009cf-3ca9-4acd-b723-b6a2eae48c2d None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:18.127834 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-608009cf-3ca9-4acd-b723-b6a2eae48c2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:18.191504 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:33:18.255527 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-e94cc76b-a79d-4bf1-8c7f-2995aaf1ffb4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3d906be03e34cb98bb5540cd91d17ef'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:18.279284 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-e94cc76b-a79d-4bf1-8c7f-2995aaf1ffb4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3d906be03e34cb98bb5540cd91d17ef'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:18.282204 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-e94cc76b-a79d-4bf1-8c7f-2995aaf1ffb4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3d906be03e34cb98bb5540cd91d17ef'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:18.286003 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-e94cc76b-a79d-4bf1-8c7f-2995aaf1ffb4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3d906be03e34cb98bb5540cd91d17ef'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:18.334021 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-e94cc76b-a79d-4bf1-8c7f-2995aaf1ffb4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3d906be03e34cb98bb5540cd91d17ef'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:18.378547 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 753/1497] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:33:17 2022] DELETE /identity/v3/users/d3d906be03e34cb98bb5540cd91d17ef => generated 0 bytes in 574 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:33:18.417972 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608009cf-3ca9-4acd-b723-b6a2eae48c2d None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:18.642512 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aadd93-e87f-4f31-88ba-f1597ed1b07c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:18.643208 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aadd93-e87f-4f31-88ba-f1597ed1b07c service neutron] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:18.643520 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aadd93-e87f-4f31-88ba-f1597ed1b07c service neutron] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:18.643796 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aadd93-e87f-4f31-88ba-f1597ed1b07c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:18.651081 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-c6aadd93-e87f-4f31-88ba-f1597ed1b07c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:18.840303 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6aadd93-e87f-4f31-88ba-f1597ed1b07c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:18.842340 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6aadd93-e87f-4f31-88ba-f1597ed1b07c service neutron] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:18.902861 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 745/1498] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:33:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 778 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 31 12:33:20.475113 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94cc76b-a79d-4bf1-8c7f-2995aaf1ffb4 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:20.477287 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-e94cc76b-a79d-4bf1-8c7f-2995aaf1ffb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:20.526865 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94cc76b-a79d-4bf1-8c7f-2995aaf1ffb4 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:20.551096 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127b79a1-5c83-4fba-aefc-26a8f9e0180a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:20.551796 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b79a1-5c83-4fba-aefc-26a8f9e0180a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:20.552078 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b79a1-5c83-4fba-aefc-26a8f9e0180a admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:20.552359 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b79a1-5c83-4fba-aefc-26a8f9e0180a admin admin] PATH_INFO: `/v3/projects/2441e30562e94e30bf3b9076ceed5f67` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:20.557481 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127b79a1-5c83-4fba-aefc-26a8f9e0180a admin admin] RBAC: Authorizing `identity:delete_project(project_id=2441e30562e94e30bf3b9076ceed5f67)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:20.567084 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127b79a1-5c83-4fba-aefc-26a8f9e0180a admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:20.719656 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-127b79a1-5c83-4fba-aefc-26a8f9e0180a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2441e30562e94e30bf3b9076ceed5f67', 'request_id': 'req-127b79a1-5c83-4fba-aefc-26a8f9e0180a'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:20.736303 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-127b79a1-5c83-4fba-aefc-26a8f9e0180a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2441e30562e94e30bf3b9076ceed5f67', 'request_id': 'req-127b79a1-5c83-4fba-aefc-26a8f9e0180a'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:20.758753 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-127b79a1-5c83-4fba-aefc-26a8f9e0180a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2441e30562e94e30bf3b9076ceed5f67', 'request_id': 'req-127b79a1-5c83-4fba-aefc-26a8f9e0180a'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:20.781895 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-127b79a1-5c83-4fba-aefc-26a8f9e0180a admin admin] The token cache is being invalidate because project 2441e30562e94e30bf3b9076ceed5f67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:33:20.782999 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-127b79a1-5c83-4fba-aefc-26a8f9e0180a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:20.784623 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-127b79a1-5c83-4fba-aefc-26a8f9e0180a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:20.787623 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 754/1499] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:33:20 2022] DELETE /identity/v3/projects/2441e30562e94e30bf3b9076ceed5f67 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:33:20.793981 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37197cfe-4f41-4462-8407-61a970f2f814 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:20.795314 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37197cfe-4f41-4462-8407-61a970f2f814 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:20.796054 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37197cfe-4f41-4462-8407-61a970f2f814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:20.856700 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-37197cfe-4f41-4462-8407-61a970f2f814 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:33:20.858384 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-37197cfe-4f41-4462-8407-61a970f2f814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:21.031598 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:33:21.160122 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 746/1500] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:33:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:33:21.162148 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f87e8e2-8b04-46b4-b9cd-0d48aa9b2ec0 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:21.162442 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f87e8e2-8b04-46b4-b9cd-0d48aa9b2ec0 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:21.162717 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f87e8e2-8b04-46b4-b9cd-0d48aa9b2ec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:21.206412 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-4f87e8e2-8b04-46b4-b9cd-0d48aa9b2ec0 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:33:21.207916 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-4f87e8e2-8b04-46b4-b9cd-0d48aa9b2ec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:21.236817 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37197cfe-4f41-4462-8407-61a970f2f814 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:21.250438 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 755/1501] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:33:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 31 12:33:21.256023 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37197cfe-4f41-4462-8407-61a970f2f814 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:21.261125 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d7d303-97ed-42d1-8e88-75d8f7c0f0b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:21.261891 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d7d303-97ed-42d1-8e88-75d8f7c0f0b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:21.262169 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d7d303-97ed-42d1-8e88-75d8f7c0f0b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:21.262435 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d7d303-97ed-42d1-8e88-75d8f7c0f0b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:21.263103 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d7d303-97ed-42d1-8e88-75d8f7c0f0b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:33:21.263477 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d7d303-97ed-42d1-8e88-75d8f7c0f0b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:21.279264 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d7d303-97ed-42d1-8e88-75d8f7c0f0b4 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:21.302576 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630716a0-c13e-487a-a1af-183350d5382a None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:21.302871 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630716a0-c13e-487a-a1af-183350d5382a None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:21.303138 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630716a0-c13e-487a-a1af-183350d5382a None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:21.305884 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 756/1502] 10.222.0.15 () {54 vars in 921 bytes} [Mon Oct 31 12:33:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 31 12:33:21.322079 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 747/1503] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Oct 31 12:33:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:33:21.322280 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630716a0-c13e-487a-a1af-183350d5382a None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:21.352367 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630716a0-c13e-487a-a1af-183350d5382a None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:21.373337 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb09108-29a5-4e04-a933-cd83938f4b80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:21.373876 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb09108-29a5-4e04-a933-cd83938f4b80 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:21.374165 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb09108-29a5-4e04-a933-cd83938f4b80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:21.374427 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb09108-29a5-4e04-a933-cd83938f4b80 admin admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:21.375052 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb09108-29a5-4e04-a933-cd83938f4b80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:21.383934 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb09108-29a5-4e04-a933-cd83938f4b80 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:21.483594 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8938a8b-4013-4cb0-af1c-eb84571fd53f None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:21.484015 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8938a8b-4013-4cb0-af1c-eb84571fd53f None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:21.484291 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8938a8b-4013-4cb0-af1c-eb84571fd53f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:21.526435 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 757/1504] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:33:21 2022] POST /identity/v3/projects => generated 407 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:33:21.530282 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb09108-29a5-4e04-a933-cd83938f4b80 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:21.543988 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-b8938a8b-4013-4cb0-af1c-eb84571fd53f None None] MFA Rules not processed for user `99889b1dab0b471d9c77887cffefc652`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:33:21.554307 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-b8938a8b-4013-4cb0-af1c-eb84571fd53f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:21.811194 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb09108-29a5-4e04-a933-cd83938f4b80 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:21.823732 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9347f90-8172-4c6c-b18c-05c89266dcff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:21.824357 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9347f90-8172-4c6c-b18c-05c89266dcff admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:21.824638 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9347f90-8172-4c6c-b18c-05c89266dcff admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:21.824908 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9347f90-8172-4c6c-b18c-05c89266dcff admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:21.825611 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9347f90-8172-4c6c-b18c-05c89266dcff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:21.827950 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 748/1505] 10.222.0.15 () {58 vars in 1018 bytes} [Mon Oct 31 12:33:21 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 345 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:33:21.831925 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9347f90-8172-4c6c-b18c-05c89266dcff admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:21.852539 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:33:21.884621 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 758/1506] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:33:21 2022] POST /identity/v3/users => generated 304 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:33:21.886659 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8938a8b-4013-4cb0-af1c-eb84571fd53f None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:21.913184 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8938a8b-4013-4cb0-af1c-eb84571fd53f None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:21.919545 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2709e56e-3919-4f0d-8d2c-c8e585c63980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:21.920422 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2709e56e-3919-4f0d-8d2c-c8e585c63980 admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:21.920756 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2709e56e-3919-4f0d-8d2c-c8e585c63980 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:21.921250 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2709e56e-3919-4f0d-8d2c-c8e585c63980 admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:21.934797 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2709e56e-3919-4f0d-8d2c-c8e585c63980 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:21.940995 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2709e56e-3919-4f0d-8d2c-c8e585c63980 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:21.953757 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 749/1507] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:33:21 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:33:21.961197 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9347f90-8172-4c6c-b18c-05c89266dcff admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:21.973148 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9347f90-8172-4c6c-b18c-05c89266dcff admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:21.979335 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58265070-6ed1-4084-92e0-6389182c28e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:21.980866 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58265070-6ed1-4084-92e0-6389182c28e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:21.981172 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58265070-6ed1-4084-92e0-6389182c28e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:21.981411 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58265070-6ed1-4084-92e0-6389182c28e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:21.982074 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58265070-6ed1-4084-92e0-6389182c28e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:21.983727 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58265070-6ed1-4084-92e0-6389182c28e8 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:22.021114 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 759/1508] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:33:21 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:33:22.032437 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2709e56e-3919-4f0d-8d2c-c8e585c63980 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:22.055133 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2709e56e-3919-4f0d-8d2c-c8e585c63980 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:22.063599 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abe9a35-1eba-4bc5-894f-d46a50c09d0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:22.064290 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abe9a35-1eba-4bc5-894f-d46a50c09d0d admin admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:22.064568 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abe9a35-1eba-4bc5-894f-d46a50c09d0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:22.064842 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abe9a35-1eba-4bc5-894f-d46a50c09d0d admin admin] PATH_INFO: `/v3/projects/ea42f548e2df4e9f8a8e72e4243950d1/users/5108166ba8d3427c8f7e7fe23fa61881/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:22.163692 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abe9a35-1eba-4bc5-894f-d46a50c09d0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea42f548e2df4e9f8a8e72e4243950d1, user_id=5108166ba8d3427c8f7e7fe23fa61881, role_id=ca1a5695f8bd499aa0b8cedaeecd2cc6)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:22.177318 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abe9a35-1eba-4bc5-894f-d46a50c09d0d admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:22.261667 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 750/1509] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:33:22 2022] PUT /identity/v3/projects/ea42f548e2df4e9f8a8e72e4243950d1/users/5108166ba8d3427c8f7e7fe23fa61881/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:33:22.276929 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48a81a5-524f-4237-a99b-d80cd4b8b455 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:22.277361 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48a81a5-524f-4237-a99b-d80cd4b8b455 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:22.277702 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48a81a5-524f-4237-a99b-d80cd4b8b455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:22.379732 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-c48a81a5-524f-4237-a99b-d80cd4b8b455 None None] MFA Rules not processed for user `5108166ba8d3427c8f7e7fe23fa61881`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:33:22.388792 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-c48a81a5-524f-4237-a99b-d80cd4b8b455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:22.638400 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abe9a35-1eba-4bc5-894f-d46a50c09d0d admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:22.656415 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 760/1510] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:33:22 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 381 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:33:22.814697 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abe9a35-1eba-4bc5-894f-d46a50c09d0d admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:22.838507 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6f72a7-ce9e-4786-b099-5e712b0b826a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:22.842065 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6f72a7-ce9e-4786-b099-5e712b0b826a admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:22.842279 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6f72a7-ce9e-4786-b099-5e712b0b826a admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:22.848651 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6f72a7-ce9e-4786-b099-5e712b0b826a admin admin] PATH_INFO: `/v3/projects` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:22.851237 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6f72a7-ce9e-4786-b099-5e712b0b826a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:22.863260 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6f72a7-ce9e-4786-b099-5e712b0b826a admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:22.950744 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 751/1511] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:33:22 2022] POST /identity/v3/projects => generated 407 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:33:22.954465 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48a81a5-524f-4237-a99b-d80cd4b8b455 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:22.968948 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48a81a5-524f-4237-a99b-d80cd4b8b455 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:23.108945 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b318357-6875-40e6-9620-19fe2b5f3c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:23.112472 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b318357-6875-40e6-9620-19fe2b5f3c54 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:23.120644 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b318357-6875-40e6-9620-19fe2b5f3c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:23.120644 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b318357-6875-40e6-9620-19fe2b5f3c54 admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:23.120644 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b318357-6875-40e6-9620-19fe2b5f3c54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:23.125648 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b318357-6875-40e6-9620-19fe2b5f3c54 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:23.163457 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 761/1512] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:33:22 2022] POST /identity/v3/users => generated 310 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:33:23.167552 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6f72a7-ce9e-4786-b099-5e712b0b826a admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:23.197518 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6f72a7-ce9e-4786-b099-5e712b0b826a admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:23.214950 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68c07f2-b22b-4957-9fe7-630e12aa114d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:23.218083 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68c07f2-b22b-4957-9fe7-630e12aa114d admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:23.219316 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68c07f2-b22b-4957-9fe7-630e12aa114d admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:23.220948 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68c07f2-b22b-4957-9fe7-630e12aa114d admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:23.233408 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68c07f2-b22b-4957-9fe7-630e12aa114d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:23.244689 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68c07f2-b22b-4957-9fe7-630e12aa114d admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:23.246923 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:33:23.289378 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 752/1513] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:33:23 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:33:23.291132 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b318357-6875-40e6-9620-19fe2b5f3c54 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:23.336203 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b318357-6875-40e6-9620-19fe2b5f3c54 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:23.347449 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c751c1f-047e-4722-91fc-034015a9c360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:23.348345 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c751c1f-047e-4722-91fc-034015a9c360 admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:23.348630 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c751c1f-047e-4722-91fc-034015a9c360 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:23.348906 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c751c1f-047e-4722-91fc-034015a9c360 admin admin] PATH_INFO: `/v3/projects/e707c2b2e76f4cdf8f41b4e9863c2e9e/users/ee460f0372324718a523c7852a85358c/roles/176c07b4fe3b4537bdfeb4be21b379c5` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:23.373411 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c751c1f-047e-4722-91fc-034015a9c360 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e707c2b2e76f4cdf8f41b4e9863c2e9e, user_id=ee460f0372324718a523c7852a85358c, role_id=176c07b4fe3b4537bdfeb4be21b379c5)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:23.378898 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c751c1f-047e-4722-91fc-034015a9c360 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:23.415182 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 762/1514] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:33:23 2022] PUT /identity/v3/projects/e707c2b2e76f4cdf8f41b4e9863c2e9e/users/ee460f0372324718a523c7852a85358c/roles/176c07b4fe3b4537bdfeb4be21b379c5 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 31 12:33:23.416792 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68c07f2-b22b-4957-9fe7-630e12aa114d admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:23.447797 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68c07f2-b22b-4957-9fe7-630e12aa114d admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:23.683723 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa292cb-57e5-4fdf-a088-bcc0486c630d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:23.684385 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa292cb-57e5-4fdf-a088-bcc0486c630d admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:23.684794 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa292cb-57e5-4fdf-a088-bcc0486c630d admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:23.684962 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa292cb-57e5-4fdf-a088-bcc0486c630d admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:23.685668 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa292cb-57e5-4fdf-a088-bcc0486c630d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:23.687318 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa292cb-57e5-4fdf-a088-bcc0486c630d admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:23.729129 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 753/1515] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:33:23 2022] GET /identity/v3/roles => generated 1420 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:33:23.733311 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c751c1f-047e-4722-91fc-034015a9c360 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:23.746863 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c751c1f-047e-4722-91fc-034015a9c360 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:23.756851 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993204a7-2d5c-4220-a26d-c456eb6d813e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:23.760259 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993204a7-2d5c-4220-a26d-c456eb6d813e admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:23.760565 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993204a7-2d5c-4220-a26d-c456eb6d813e admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:23.760840 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993204a7-2d5c-4220-a26d-c456eb6d813e admin admin] PATH_INFO: `/v3/projects/e707c2b2e76f4cdf8f41b4e9863c2e9e/users/ee460f0372324718a523c7852a85358c/roles/176c07b4fe3b4537bdfeb4be21b379c5` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:23.763971 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993204a7-2d5c-4220-a26d-c456eb6d813e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e707c2b2e76f4cdf8f41b4e9863c2e9e, user_id=ee460f0372324718a523c7852a85358c, role_id=176c07b4fe3b4537bdfeb4be21b379c5)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:23.773911 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993204a7-2d5c-4220-a26d-c456eb6d813e admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:23.806230 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 763/1516] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:33:23 2022] PUT /identity/v3/projects/e707c2b2e76f4cdf8f41b4e9863c2e9e/users/ee460f0372324718a523c7852a85358c/roles/176c07b4fe3b4537bdfeb4be21b379c5 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:33:23.819871 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d127172-7618-412b-b566-d8ce2a22a536 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:23.820522 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d127172-7618-412b-b566-d8ce2a22a536 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:23.821043 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d127172-7618-412b-b566-d8ce2a22a536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:23.882087 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-4d127172-7618-412b-b566-d8ce2a22a536 None None] MFA Rules not processed for user `ee460f0372324718a523c7852a85358c`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:33:23.887473 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-4d127172-7618-412b-b566-d8ce2a22a536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:23.958938 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993204a7-2d5c-4220-a26d-c456eb6d813e admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:23.961169 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-993204a7-2d5c-4220-a26d-c456eb6d813e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:24.051897 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:33:24.158112 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993204a7-2d5c-4220-a26d-c456eb6d813e admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:24.160184 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 754/1517] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:33:23 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 342 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:33:24.171388 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d127172-7618-412b-b566-d8ce2a22a536 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:24.173163 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-4d127172-7618-412b-b566-d8ce2a22a536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:24.174892 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bc5b9c-8d78-4fad-bff0-9bbfa7c502fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99889b1dab0b471d9c77887cffefc652', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:24.176906 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bc5b9c-8d78-4fad-bff0-9bbfa7c502fe service nova] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:24.177936 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bc5b9c-8d78-4fad-bff0-9bbfa7c502fe service nova] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:24.179037 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bc5b9c-8d78-4fad-bff0-9bbfa7c502fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:24.213773 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bc5b9c-8d78-4fad-bff0-9bbfa7c502fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:24.215622 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bc5b9c-8d78-4fad-bff0-9bbfa7c502fe service nova] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:24.236457 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 764/1518] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:33:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 31 12:33:24.259339 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bc5b9c-8d78-4fad-bff0-9bbfa7c502fe service nova] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:24.260175 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-c8bc5b9c-8d78-4fad-bff0-9bbfa7c502fe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:24.323628 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d127172-7618-412b-b566-d8ce2a22a536 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:24.361010 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfcf8d2-6dd1-424e-9156-ff64df71006c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d6d7ea077d49e8a66c6c6762499702', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:24.364358 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfcf8d2-6dd1-424e-9156-ff64df71006c service cinder] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:24.364646 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfcf8d2-6dd1-424e-9156-ff64df71006c service cinder] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:24.366147 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfcf8d2-6dd1-424e-9156-ff64df71006c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:24.399031 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfcf8d2-6dd1-424e-9156-ff64df71006c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:24.413979 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfcf8d2-6dd1-424e-9156-ff64df71006c service cinder] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:24.450017 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bc5b9c-8d78-4fad-bff0-9bbfa7c502fe service nova] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:24.463199 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 755/1519] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:33:24 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 31 12:33:24.465324 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08a933d-39ce-403e-99a1-4eafb7748f69 None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:24.466037 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08a933d-39ce-403e-99a1-4eafb7748f69 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:24.468404 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08a933d-39ce-403e-99a1-4eafb7748f69 None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:24.469828 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee53461-28c0-47a1-9013-c74b85103117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:24.472338 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee53461-28c0-47a1-9013-c74b85103117 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:24.473120 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee53461-28c0-47a1-9013-c74b85103117 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:24.478349 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 756/1520] 10.222.0.15 () {54 vars in 921 bytes} [Mon Oct 31 12:33:24 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:33:24.479373 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee53461-28c0-47a1-9013-c74b85103117 admin admin] PATH_INFO: `/v3/projects/c3b3fabfa5054f74928712bae5020428` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:24.483867 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee53461-28c0-47a1-9013-c74b85103117 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3b3fabfa5054f74928712bae5020428)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:24.488770 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee53461-28c0-47a1-9013-c74b85103117 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:24.553277 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08a933d-39ce-403e-99a1-4eafb7748f69 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:24.566776 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-b08a933d-39ce-403e-99a1-4eafb7748f69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:24.615980 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08a933d-39ce-403e-99a1-4eafb7748f69 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:24.649237 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa6514d-bc66-4ca1-9d88-5031b0782868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:24.649895 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa6514d-bc66-4ca1-9d88-5031b0782868 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:24.652072 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa6514d-bc66-4ca1-9d88-5031b0782868 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:24.652429 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa6514d-bc66-4ca1-9d88-5031b0782868 admin admin] PATH_INFO: `/v3/users/8efc04798db9441cb1c461500b3fb03d` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:24.654649 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa6514d-bc66-4ca1-9d88-5031b0782868 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8efc04798db9441cb1c461500b3fb03d)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:24.656626 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa6514d-bc66-4ca1-9d88-5031b0782868 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:24.708475 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-0ee53461-28c0-47a1-9013-c74b85103117 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3b3fabfa5054f74928712bae5020428', 'request_id': 'req-0ee53461-28c0-47a1-9013-c74b85103117'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:24.727851 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:33:24.782006 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-0ee53461-28c0-47a1-9013-c74b85103117 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3b3fabfa5054f74928712bae5020428', 'request_id': 'req-0ee53461-28c0-47a1-9013-c74b85103117'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:24.808344 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-0ee53461-28c0-47a1-9013-c74b85103117 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3b3fabfa5054f74928712bae5020428', 'request_id': 'req-0ee53461-28c0-47a1-9013-c74b85103117'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:24.856174 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-0ee53461-28c0-47a1-9013-c74b85103117 admin admin] The token cache is being invalidate because project c3b3fabfa5054f74928712bae5020428 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:33:24.863692 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-0ee53461-28c0-47a1-9013-c74b85103117 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:24.866306 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-5aa6514d-bc66-4ca1-9d88-5031b0782868 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8efc04798db9441cb1c461500b3fb03d'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:24.866584 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-0ee53461-28c0-47a1-9013-c74b85103117 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:24.875309 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 765/1521] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:33:24 2022] DELETE /identity/v3/projects/c3b3fabfa5054f74928712bae5020428 => generated 0 bytes in 614 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:33:24.881933 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-5aa6514d-bc66-4ca1-9d88-5031b0782868 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8efc04798db9441cb1c461500b3fb03d'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:24.881933 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-5aa6514d-bc66-4ca1-9d88-5031b0782868 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8efc04798db9441cb1c461500b3fb03d'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:24.884121 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b7f582-e3c5-4b23-b331-6404c69964c4 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:24.884414 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b7f582-e3c5-4b23-b331-6404c69964c4 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:24.885733 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b7f582-e3c5-4b23-b331-6404c69964c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:24.929773 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-5aa6514d-bc66-4ca1-9d88-5031b0782868 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8efc04798db9441cb1c461500b3fb03d'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:24.960626 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-5aa6514d-bc66-4ca1-9d88-5031b0782868 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8efc04798db9441cb1c461500b3fb03d'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:24.976997 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 757/1522] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:33:24 2022] DELETE /identity/v3/users/8efc04798db9441cb1c461500b3fb03d => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:33:24.995725 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa6514d-bc66-4ca1-9d88-5031b0782868 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:25.004224 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-5aa6514d-bc66-4ca1-9d88-5031b0782868 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:25.044113 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-a4b7f582-e3c5-4b23-b331-6404c69964c4 None None] MFA Rules not processed for user `99889b1dab0b471d9c77887cffefc652`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:33:25.045633 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-a4b7f582-e3c5-4b23-b331-6404c69964c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:25.121465 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:33:25.209530 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 766/1523] 10.222.0.15 () {58 vars in 1018 bytes} [Mon Oct 31 12:33:24 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:33:25.210759 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db083466-5388-40ee-acba-e14bf1a73e7d None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:25.211087 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db083466-5388-40ee-acba-e14bf1a73e7d None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:25.211385 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db083466-5388-40ee-acba-e14bf1a73e7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:25.230483 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa6514d-bc66-4ca1-9d88-5031b0782868 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:25.243969 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d488f6-a4ca-4af8-b3d9-085932fc3db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:25.245056 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-db083466-5388-40ee-acba-e14bf1a73e7d None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:33:25.245222 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d488f6-a4ca-4af8-b3d9-085932fc3db2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:25.245638 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d488f6-a4ca-4af8-b3d9-085932fc3db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:25.246324 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-db083466-5388-40ee-acba-e14bf1a73e7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:25.247364 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d488f6-a4ca-4af8-b3d9-085932fc3db2 admin admin] PATH_INFO: `/v3/users/d610922674754b598ace753064c1fb8f` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:25.253572 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d488f6-a4ca-4af8-b3d9-085932fc3db2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d610922674754b598ace753064c1fb8f)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:25.257671 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d488f6-a4ca-4af8-b3d9-085932fc3db2 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:25.264817 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 767/1524] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:33:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:33:25.265902 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db083466-5388-40ee-acba-e14bf1a73e7d None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:25.271518 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-db083466-5388-40ee-acba-e14bf1a73e7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:25.362403 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-52d488f6-a4ca-4af8-b3d9-085932fc3db2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd610922674754b598ace753064c1fb8f'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:25.374560 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-52d488f6-a4ca-4af8-b3d9-085932fc3db2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd610922674754b598ace753064c1fb8f'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:25.375477 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-52d488f6-a4ca-4af8-b3d9-085932fc3db2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd610922674754b598ace753064c1fb8f'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:25.401850 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db083466-5388-40ee-acba-e14bf1a73e7d None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:25.407284 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97761c58-e1af-4826-8dd8-900e326bff2c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d6d7ea077d49e8a66c6c6762499702', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:25.408213 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97761c58-e1af-4826-8dd8-900e326bff2c service cinder] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:25.408704 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97761c58-e1af-4826-8dd8-900e326bff2c service cinder] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:25.410200 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97761c58-e1af-4826-8dd8-900e326bff2c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:25.412751 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-97761c58-e1af-4826-8dd8-900e326bff2c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:25.419749 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-52d488f6-a4ca-4af8-b3d9-085932fc3db2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd610922674754b598ace753064c1fb8f'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:25.455675 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-52d488f6-a4ca-4af8-b3d9-085932fc3db2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd610922674754b598ace753064c1fb8f'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:25.469792 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 758/1525] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:33:24 2022] DELETE /identity/v3/users/d610922674754b598ace753064c1fb8f => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:33:25.470521 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d488f6-a4ca-4af8-b3d9-085932fc3db2 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:25.473323 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-52d488f6-a4ca-4af8-b3d9-085932fc3db2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:25.490419 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97761c58-e1af-4826-8dd8-900e326bff2c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:25.492072 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97761c58-e1af-4826-8dd8-900e326bff2c service cinder] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:25.497869 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:33:25.503220 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 768/1526] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:33:25 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:33:25.592970 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d488f6-a4ca-4af8-b3d9-085932fc3db2 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:25.598644 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d305f97-bcc3-41f2-a92e-854c940e17fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:25.600598 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d305f97-bcc3-41f2-a92e-854c940e17fc service neutron] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:25.600598 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d305f97-bcc3-41f2-a92e-854c940e17fc service neutron] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:25.600598 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d305f97-bcc3-41f2-a92e-854c940e17fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:25.604190 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-3d305f97-bcc3-41f2-a92e-854c940e17fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:25.736095 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97761c58-e1af-4826-8dd8-900e326bff2c service cinder] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:25.736095 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-97761c58-e1af-4826-8dd8-900e326bff2c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:25.761560 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97761c58-e1af-4826-8dd8-900e326bff2c service cinder] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:25.762752 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d305f97-bcc3-41f2-a92e-854c940e17fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:25.765644 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d305f97-bcc3-41f2-a92e-854c940e17fc service neutron] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:25.786568 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 759/1527] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:33:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:33:25.890865 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce035de2-df3a-480f-b850-fab05ce34172 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99889b1dab0b471d9c77887cffefc652', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:25.891832 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce035de2-df3a-480f-b850-fab05ce34172 service nova] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:25.892118 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce035de2-df3a-480f-b850-fab05ce34172 service nova] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:25.892433 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce035de2-df3a-480f-b850-fab05ce34172 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:25.896124 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-ce035de2-df3a-480f-b850-fab05ce34172 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:25.915488 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce035de2-df3a-480f-b850-fab05ce34172 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:25.934510 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce035de2-df3a-480f-b850-fab05ce34172 service nova] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:25.944781 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:33:25.966155 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 769/1528] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:33:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:33:26.610129 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d305f97-bcc3-41f2-a92e-854c940e17fc service neutron] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:26.611570 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-3d305f97-bcc3-41f2-a92e-854c940e17fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:26.631126 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d305f97-bcc3-41f2-a92e-854c940e17fc service neutron] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:26.635393 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9e0f09-9f18-4910-a1c2-8e8247cef23d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:26.636321 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9e0f09-9f18-4910-a1c2-8e8247cef23d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:26.636608 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9e0f09-9f18-4910-a1c2-8e8247cef23d admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:26.636880 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9e0f09-9f18-4910-a1c2-8e8247cef23d admin admin] PATH_INFO: `/v3/projects/d549fbb3c6bf4014b6fc54ab5b45386d` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:26.641263 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9e0f09-9f18-4910-a1c2-8e8247cef23d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d549fbb3c6bf4014b6fc54ab5b45386d)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:26.654628 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9e0f09-9f18-4910-a1c2-8e8247cef23d admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:26.757782 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-9c9e0f09-9f18-4910-a1c2-8e8247cef23d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd549fbb3c6bf4014b6fc54ab5b45386d', 'request_id': 'req-9c9e0f09-9f18-4910-a1c2-8e8247cef23d'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:26.766743 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-9c9e0f09-9f18-4910-a1c2-8e8247cef23d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd549fbb3c6bf4014b6fc54ab5b45386d', 'request_id': 'req-9c9e0f09-9f18-4910-a1c2-8e8247cef23d'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:26.777721 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-9c9e0f09-9f18-4910-a1c2-8e8247cef23d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd549fbb3c6bf4014b6fc54ab5b45386d', 'request_id': 'req-9c9e0f09-9f18-4910-a1c2-8e8247cef23d'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:26.798964 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-9c9e0f09-9f18-4910-a1c2-8e8247cef23d admin admin] The token cache is being invalidate because project d549fbb3c6bf4014b6fc54ab5b45386d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:33:26.798964 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-9c9e0f09-9f18-4910-a1c2-8e8247cef23d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:26.800066 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-9c9e0f09-9f18-4910-a1c2-8e8247cef23d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:26.804808 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 760/1529] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:33:26 2022] DELETE /identity/v3/projects/d549fbb3c6bf4014b6fc54ab5b45386d => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:33:26.806635 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e0ece6-19e1-471d-b8a5-fe1729fd833d None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:26.807006 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e0ece6-19e1-471d-b8a5-fe1729fd833d None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:26.809838 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e0ece6-19e1-471d-b8a5-fe1729fd833d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:26.829420 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-a6e0ece6-19e1-471d-b8a5-fe1729fd833d None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:33:26.831549 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-a6e0ece6-19e1-471d-b8a5-fe1729fd833d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:26.904175 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 770/1530] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:33:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:33:26.911600 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35798abb-c6c6-494d-92e0-449de7c3dda1 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:26.912014 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35798abb-c6c6-494d-92e0-449de7c3dda1 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:26.912387 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35798abb-c6c6-494d-92e0-449de7c3dda1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:26.940719 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-35798abb-c6c6-494d-92e0-449de7c3dda1 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:33:26.940719 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-35798abb-c6c6-494d-92e0-449de7c3dda1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:26.949268 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 761/1531] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:33:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:33:26.954576 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e0ece6-19e1-471d-b8a5-fe1729fd833d None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:26.968897 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e0ece6-19e1-471d-b8a5-fe1729fd833d None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:26.975557 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc8dabd-3f3b-4a1f-b7b9-cb7ee337fc2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:26.977067 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc8dabd-3f3b-4a1f-b7b9-cb7ee337fc2e admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:26.977351 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc8dabd-3f3b-4a1f-b7b9-cb7ee337fc2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:26.977623 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc8dabd-3f3b-4a1f-b7b9-cb7ee337fc2e admin admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:26.978278 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc8dabd-3f3b-4a1f-b7b9-cb7ee337fc2e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:33:26.978579 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc8dabd-3f3b-4a1f-b7b9-cb7ee337fc2e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:26.980216 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc8dabd-3f3b-4a1f-b7b9-cb7ee337fc2e admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:27.004734 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 771/1532] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Oct 31 12:33:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:33:27.005609 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35798abb-c6c6-494d-92e0-449de7c3dda1 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:27.037863 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35798abb-c6c6-494d-92e0-449de7c3dda1 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:27.043635 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a7324b-ab34-47f9-ab19-9b81162f4628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:27.044404 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a7324b-ab34-47f9-ab19-9b81162f4628 admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:27.044685 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a7324b-ab34-47f9-ab19-9b81162f4628 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:27.044958 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a7324b-ab34-47f9-ab19-9b81162f4628 admin admin] PATH_INFO: `/v3/projects` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:27.045600 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a7324b-ab34-47f9-ab19-9b81162f4628 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:27.047947 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a7324b-ab34-47f9-ab19-9b81162f4628 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:27.087275 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 762/1533] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:33:27 2022] POST /identity/v3/projects => generated 385 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:33:27.091708 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc8dabd-3f3b-4a1f-b7b9-cb7ee337fc2e admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:27.110741 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc8dabd-3f3b-4a1f-b7b9-cb7ee337fc2e admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:27.209828 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b49e22-d2c3-4792-a9f7-8296909d2eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:27.210241 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b49e22-d2c3-4792-a9f7-8296909d2eb3 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:27.210520 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b49e22-d2c3-4792-a9f7-8296909d2eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:27.210821 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b49e22-d2c3-4792-a9f7-8296909d2eb3 admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:27.211445 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b49e22-d2c3-4792-a9f7-8296909d2eb3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:27.213425 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b49e22-d2c3-4792-a9f7-8296909d2eb3 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:27.240690 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 772/1534] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:33:27 2022] POST /identity/v3/users => generated 293 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:33:27.244854 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a7324b-ab34-47f9-ab19-9b81162f4628 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:27.259480 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a7324b-ab34-47f9-ab19-9b81162f4628 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:27.268807 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f19afc4-67a4-4316-a3e3-64fb47beaa31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:27.269709 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f19afc4-67a4-4316-a3e3-64fb47beaa31 admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:27.271229 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f19afc4-67a4-4316-a3e3-64fb47beaa31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:27.271561 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f19afc4-67a4-4316-a3e3-64fb47beaa31 admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:27.272530 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f19afc4-67a4-4316-a3e3-64fb47beaa31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:27.274366 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f19afc4-67a4-4316-a3e3-64fb47beaa31 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:27.295593 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b49e22-d2c3-4792-a9f7-8296909d2eb3 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:27.297905 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-48b49e22-d2c3-4792-a9f7-8296909d2eb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:27.307080 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 763/1535] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:33:27 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 31 12:33:27.311820 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f19afc4-67a4-4316-a3e3-64fb47beaa31 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:27.320602 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b49e22-d2c3-4792-a9f7-8296909d2eb3 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:27.325733 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:33:27.331457 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be1e355-077a-440b-9f74-7f717d91a594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:27.331730 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be1e355-077a-440b-9f74-7f717d91a594 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:27.332014 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be1e355-077a-440b-9f74-7f717d91a594 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:27.332281 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be1e355-077a-440b-9f74-7f717d91a594 admin admin] PATH_INFO: `/v3/projects/62d591a4cc824393886ced9e459a8002` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:27.335003 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f19afc4-67a4-4316-a3e3-64fb47beaa31 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:27.335690 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be1e355-077a-440b-9f74-7f717d91a594 admin admin] RBAC: Authorizing `identity:delete_project(project_id=62d591a4cc824393886ced9e459a8002)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:27.338973 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be1e355-077a-440b-9f74-7f717d91a594 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:27.345434 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863f905f-7cc3-4eca-a18b-50e7f5d948f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:27.347655 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863f905f-7cc3-4eca-a18b-50e7f5d948f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:27.348576 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863f905f-7cc3-4eca-a18b-50e7f5d948f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:27.348874 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863f905f-7cc3-4eca-a18b-50e7f5d948f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:27.349526 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863f905f-7cc3-4eca-a18b-50e7f5d948f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:27.351388 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863f905f-7cc3-4eca-a18b-50e7f5d948f1 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:27.375402 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 764/1536] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:33:27 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:33:27.380218 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863f905f-7cc3-4eca-a18b-50e7f5d948f1 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:27.417336 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863f905f-7cc3-4eca-a18b-50e7f5d948f1 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:27.459107 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-0be1e355-077a-440b-9f74-7f717d91a594 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62d591a4cc824393886ced9e459a8002', 'request_id': 'req-0be1e355-077a-440b-9f74-7f717d91a594'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:27.482649 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-0be1e355-077a-440b-9f74-7f717d91a594 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62d591a4cc824393886ced9e459a8002', 'request_id': 'req-0be1e355-077a-440b-9f74-7f717d91a594'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:27.499531 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-0be1e355-077a-440b-9f74-7f717d91a594 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62d591a4cc824393886ced9e459a8002', 'request_id': 'req-0be1e355-077a-440b-9f74-7f717d91a594'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:27.510550 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975f93c7-ec1a-4ba6-beb8-c8723f1a7001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:27.511388 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:33:27.511523 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975f93c7-ec1a-4ba6-beb8-c8723f1a7001 admin admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:27.511824 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975f93c7-ec1a-4ba6-beb8-c8723f1a7001 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:27.512162 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975f93c7-ec1a-4ba6-beb8-c8723f1a7001 admin admin] PATH_INFO: `/v3/projects/d783018f8c854d42a147376a405f2f7f/users/e7787006fb13497c82938b1f9032ad53/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:27.517388 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-0be1e355-077a-440b-9f74-7f717d91a594 admin admin] The token cache is being invalidate because project 62d591a4cc824393886ced9e459a8002 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:33:27.517743 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-0be1e355-077a-440b-9f74-7f717d91a594 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:27.520017 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-0be1e355-077a-440b-9f74-7f717d91a594 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:27.522137 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 773/1537] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:33:27 2022] DELETE /identity/v3/projects/62d591a4cc824393886ced9e459a8002 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 31 12:33:27.526974 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be1e355-077a-440b-9f74-7f717d91a594 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:27.528792 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-0be1e355-077a-440b-9f74-7f717d91a594 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:27.534013 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975f93c7-ec1a-4ba6-beb8-c8723f1a7001 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d783018f8c854d42a147376a405f2f7f, user_id=e7787006fb13497c82938b1f9032ad53, role_id=ca1a5695f8bd499aa0b8cedaeecd2cc6)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:27.537413 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975f93c7-ec1a-4ba6-beb8-c8723f1a7001 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:27.547683 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be1e355-077a-440b-9f74-7f717d91a594 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:27.555268 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 765/1538] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:33:27 2022] PUT /identity/v3/projects/d783018f8c854d42a147376a405f2f7f/users/e7787006fb13497c82938b1f9032ad53/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:33:27.566804 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88b988f-27e8-4642-86ff-d510b565837b None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:27.567090 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88b988f-27e8-4642-86ff-d510b565837b None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:27.567402 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88b988f-27e8-4642-86ff-d510b565837b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:27.604939 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-c88b988f-27e8-4642-86ff-d510b565837b None None] MFA Rules not processed for user `e7787006fb13497c82938b1f9032ad53`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:33:27.608747 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-c88b988f-27e8-4642-86ff-d510b565837b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:27.690764 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb3b7ea-3526-48e7-9250-5ccaef9f0392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:27.691740 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb3b7ea-3526-48e7-9250-5ccaef9f0392 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:27.692014 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb3b7ea-3526-48e7-9250-5ccaef9f0392 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:27.692321 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb3b7ea-3526-48e7-9250-5ccaef9f0392 admin admin] PATH_INFO: `/v3/users/0b8d3d4975c145e1adb7886c0539f918` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:27.694837 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb3b7ea-3526-48e7-9250-5ccaef9f0392 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b8d3d4975c145e1adb7886c0539f918)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:27.697627 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb3b7ea-3526-48e7-9250-5ccaef9f0392 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:27.777632 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 766/1539] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:33:27 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 31 12:33:27.787123 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88b988f-27e8-4642-86ff-d510b565837b None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:27.789298 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-c88b988f-27e8-4642-86ff-d510b565837b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:27.801369 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-6cb3b7ea-3526-48e7-9250-5ccaef9f0392 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b8d3d4975c145e1adb7886c0539f918'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:27.815027 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-6cb3b7ea-3526-48e7-9250-5ccaef9f0392 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b8d3d4975c145e1adb7886c0539f918'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:27.816843 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-6cb3b7ea-3526-48e7-9250-5ccaef9f0392 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b8d3d4975c145e1adb7886c0539f918'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:27.818263 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-6cb3b7ea-3526-48e7-9250-5ccaef9f0392 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b8d3d4975c145e1adb7886c0539f918'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:27.843666 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-6cb3b7ea-3526-48e7-9250-5ccaef9f0392 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b8d3d4975c145e1adb7886c0539f918'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:27.867696 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 774/1540] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:33:27 2022] DELETE /identity/v3/users/0b8d3d4975c145e1adb7886c0539f918 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:33:27.887011 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88b988f-27e8-4642-86ff-d510b565837b None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:27.907609 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:33:27.949576 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eac174-870f-40d1-9f18-fe79b2e2d36e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d6d7ea077d49e8a66c6c6762499702', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:27.950192 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eac174-870f-40d1-9f18-fe79b2e2d36e service cinder] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:27.950470 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eac174-870f-40d1-9f18-fe79b2e2d36e service cinder] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:27.950770 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eac174-870f-40d1-9f18-fe79b2e2d36e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:27.952737 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-78eac174-870f-40d1-9f18-fe79b2e2d36e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:28.032018 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78eac174-870f-40d1-9f18-fe79b2e2d36e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:28.034404 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78eac174-870f-40d1-9f18-fe79b2e2d36e service cinder] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:28.048350 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 767/1541] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:33:27 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 31 12:33:28.083217 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb3b7ea-3526-48e7-9250-5ccaef9f0392 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:28.085025 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-6cb3b7ea-3526-48e7-9250-5ccaef9f0392 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:28.223756 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb3b7ea-3526-48e7-9250-5ccaef9f0392 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:28.232362 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501d74e6-af6f-4b45-927c-b2fc7b21cda6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:28.232942 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501d74e6-af6f-4b45-927c-b2fc7b21cda6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:28.233220 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501d74e6-af6f-4b45-927c-b2fc7b21cda6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:28.234157 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501d74e6-af6f-4b45-927c-b2fc7b21cda6 admin admin] PATH_INFO: `/v3/users/e7787006fb13497c82938b1f9032ad53` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:28.236368 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501d74e6-af6f-4b45-927c-b2fc7b21cda6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7787006fb13497c82938b1f9032ad53)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:28.239937 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501d74e6-af6f-4b45-927c-b2fc7b21cda6 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:28.274178 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:33:28.319048 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-501d74e6-af6f-4b45-927c-b2fc7b21cda6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7787006fb13497c82938b1f9032ad53'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:28.333005 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-501d74e6-af6f-4b45-927c-b2fc7b21cda6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7787006fb13497c82938b1f9032ad53'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:28.341067 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-501d74e6-af6f-4b45-927c-b2fc7b21cda6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7787006fb13497c82938b1f9032ad53'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:28.342076 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-501d74e6-af6f-4b45-927c-b2fc7b21cda6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7787006fb13497c82938b1f9032ad53'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:28.355292 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-501d74e6-af6f-4b45-927c-b2fc7b21cda6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7787006fb13497c82938b1f9032ad53'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:28.368864 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 775/1542] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:33:28 2022] DELETE /identity/v3/users/e7787006fb13497c82938b1f9032ad53 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:33:28.373457 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973141c0-35db-4034-bbcd-f824837ab6ec None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:28.373807 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973141c0-35db-4034-bbcd-f824837ab6ec None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:28.374145 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973141c0-35db-4034-bbcd-f824837ab6ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:28.394191 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-973141c0-35db-4034-bbcd-f824837ab6ec None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:33:28.395799 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-973141c0-35db-4034-bbcd-f824837ab6ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:28.460928 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 768/1543] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:33:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:33:28.493772 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501d74e6-af6f-4b45-927c-b2fc7b21cda6 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:28.495677 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-501d74e6-af6f-4b45-927c-b2fc7b21cda6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:28.608028 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973141c0-35db-4034-bbcd-f824837ab6ec None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:28.608028 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-973141c0-35db-4034-bbcd-f824837ab6ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:28.611046 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501d74e6-af6f-4b45-927c-b2fc7b21cda6 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:28.615499 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2791c661-738e-4eab-8ada-e4aa0a86fa13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:28.616110 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2791c661-738e-4eab-8ada-e4aa0a86fa13 service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:28.616514 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2791c661-738e-4eab-8ada-e4aa0a86fa13 service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:28.616784 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2791c661-738e-4eab-8ada-e4aa0a86fa13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:28.620474 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973141c0-35db-4034-bbcd-f824837ab6ec None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:28.625427 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64f3938-b907-4c46-ab41-f5961f27ea6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:28.626067 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64f3938-b907-4c46-ab41-f5961f27ea6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:28.626347 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64f3938-b907-4c46-ab41-f5961f27ea6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:28.626615 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64f3938-b907-4c46-ab41-f5961f27ea6d admin admin] PATH_INFO: `/v3/projects/685a10b3658849e2b21e61ca1222bda6` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:28.627978 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64f3938-b907-4c46-ab41-f5961f27ea6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=685a10b3658849e2b21e61ca1222bda6)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:28.630076 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64f3938-b907-4c46-ab41-f5961f27ea6d admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:28.633361 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2791c661-738e-4eab-8ada-e4aa0a86fa13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:28.635030 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2791c661-738e-4eab-8ada-e4aa0a86fa13 service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:28.653053 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 776/1544] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:33:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:33:28.693742 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-b64f3938-b907-4c46-ab41-f5961f27ea6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '685a10b3658849e2b21e61ca1222bda6', 'request_id': 'req-b64f3938-b907-4c46-ab41-f5961f27ea6d'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:28.701120 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-b64f3938-b907-4c46-ab41-f5961f27ea6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '685a10b3658849e2b21e61ca1222bda6', 'request_id': 'req-b64f3938-b907-4c46-ab41-f5961f27ea6d'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:28.710895 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-b64f3938-b907-4c46-ab41-f5961f27ea6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '685a10b3658849e2b21e61ca1222bda6', 'request_id': 'req-b64f3938-b907-4c46-ab41-f5961f27ea6d'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:28.723310 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:33:28.728681 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-b64f3938-b907-4c46-ab41-f5961f27ea6d admin admin] The token cache is being invalidate because project 685a10b3658849e2b21e61ca1222bda6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:33:28.729360 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-b64f3938-b907-4c46-ab41-f5961f27ea6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:28.730936 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-b64f3938-b907-4c46-ab41-f5961f27ea6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:28.734072 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 769/1545] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:33:28 2022] DELETE /identity/v3/projects/685a10b3658849e2b21e61ca1222bda6 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:33:28.738224 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4daddc-1d50-4584-8b31-969c8ec2da48 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:28.738667 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4daddc-1d50-4584-8b31-969c8ec2da48 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:28.739139 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4daddc-1d50-4584-8b31-969c8ec2da48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:28.763845 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-1d4daddc-1d50-4584-8b31-969c8ec2da48 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:33:28.765374 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-1d4daddc-1d50-4584-8b31-969c8ec2da48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:28.843145 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 777/1546] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:33:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:33:28.851260 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e29f233-ef90-45b5-ac42-9324a0138b47 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:28.851752 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e29f233-ef90-45b5-ac42-9324a0138b47 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:28.852029 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e29f233-ef90-45b5-ac42-9324a0138b47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:28.872224 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-4e29f233-ef90-45b5-ac42-9324a0138b47 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:33:28.873343 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-4e29f233-ef90-45b5-ac42-9324a0138b47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:28.882328 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 770/1547] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:33:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:33:28.886254 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4daddc-1d50-4584-8b31-969c8ec2da48 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:28.897906 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4daddc-1d50-4584-8b31-969c8ec2da48 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:28.902706 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29864b54-5f1e-4245-929a-58a4b78a313b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:28.903451 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29864b54-5f1e-4245-929a-58a4b78a313b admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:28.903801 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29864b54-5f1e-4245-929a-58a4b78a313b admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:28.904149 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29864b54-5f1e-4245-929a-58a4b78a313b admin admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:28.904859 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29864b54-5f1e-4245-929a-58a4b78a313b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:33:28.905234 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29864b54-5f1e-4245-929a-58a4b78a313b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:28.907145 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29864b54-5f1e-4245-929a-58a4b78a313b admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:28.920261 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 778/1548] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Oct 31 12:33:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:33:28.924682 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e29f233-ef90-45b5-ac42-9324a0138b47 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:28.934701 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e29f233-ef90-45b5-ac42-9324a0138b47 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:28.940597 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc5400a-8177-4bdf-a5a6-9abfebc86757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:28.941328 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc5400a-8177-4bdf-a5a6-9abfebc86757 admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:28.941654 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc5400a-8177-4bdf-a5a6-9abfebc86757 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:28.942040 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc5400a-8177-4bdf-a5a6-9abfebc86757 admin admin] PATH_INFO: `/v3/projects` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:28.942814 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc5400a-8177-4bdf-a5a6-9abfebc86757 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:28.944963 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc5400a-8177-4bdf-a5a6-9abfebc86757 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:28.974868 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 771/1549] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:33:28 2022] POST /identity/v3/projects => generated 377 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:33:28.979176 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29864b54-5f1e-4245-929a-58a4b78a313b admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:28.989517 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29864b54-5f1e-4245-929a-58a4b78a313b admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:29.015619 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:33:29.064610 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc5400a-8177-4bdf-a5a6-9abfebc86757 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:29.066540 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-dbc5400a-8177-4bdf-a5a6-9abfebc86757 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:29.070859 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eec0d46-ee1d-48db-b540-140c383015ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:29.071489 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eec0d46-ee1d-48db-b540-140c383015ff admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:29.071836 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eec0d46-ee1d-48db-b540-140c383015ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:29.072113 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eec0d46-ee1d-48db-b540-140c383015ff admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:29.072750 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eec0d46-ee1d-48db-b540-140c383015ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:29.074838 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eec0d46-ee1d-48db-b540-140c383015ff admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:29.082088 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc5400a-8177-4bdf-a5a6-9abfebc86757 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:29.088003 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968145fa-43e2-4ebd-bd84-9e729c30256f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:29.088994 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968145fa-43e2-4ebd-bd84-9e729c30256f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:29.089383 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968145fa-43e2-4ebd-bd84-9e729c30256f admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:29.089748 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968145fa-43e2-4ebd-bd84-9e729c30256f admin admin] PATH_INFO: `/v3/projects/d783018f8c854d42a147376a405f2f7f` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:29.091634 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968145fa-43e2-4ebd-bd84-9e729c30256f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d783018f8c854d42a147376a405f2f7f)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:29.094200 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968145fa-43e2-4ebd-bd84-9e729c30256f admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:29.104241 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 779/1550] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:33:28 2022] POST /identity/v3/users => generated 289 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 31 12:33:29.108306 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eec0d46-ee1d-48db-b540-140c383015ff admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:29.124635 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eec0d46-ee1d-48db-b540-140c383015ff admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:29.140570 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63845262-f2ed-4fc7-8583-4925d0b3f2db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:29.141232 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63845262-f2ed-4fc7-8583-4925d0b3f2db admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:29.141520 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63845262-f2ed-4fc7-8583-4925d0b3f2db admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:29.141792 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63845262-f2ed-4fc7-8583-4925d0b3f2db admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:29.142724 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63845262-f2ed-4fc7-8583-4925d0b3f2db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:29.145238 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63845262-f2ed-4fc7-8583-4925d0b3f2db admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:29.149887 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-968145fa-43e2-4ebd-bd84-9e729c30256f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd783018f8c854d42a147376a405f2f7f', 'request_id': 'req-968145fa-43e2-4ebd-bd84-9e729c30256f'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:29.156113 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-968145fa-43e2-4ebd-bd84-9e729c30256f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd783018f8c854d42a147376a405f2f7f', 'request_id': 'req-968145fa-43e2-4ebd-bd84-9e729c30256f'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:29.157058 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 780/1551] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:33:29 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:33:29.161118 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63845262-f2ed-4fc7-8583-4925d0b3f2db admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:29.165894 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-968145fa-43e2-4ebd-bd84-9e729c30256f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd783018f8c854d42a147376a405f2f7f', 'request_id': 'req-968145fa-43e2-4ebd-bd84-9e729c30256f'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:29.172593 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63845262-f2ed-4fc7-8583-4925d0b3f2db admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:29.183930 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-968145fa-43e2-4ebd-bd84-9e729c30256f admin admin] The token cache is being invalidate because project d783018f8c854d42a147376a405f2f7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:33:29.184544 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-968145fa-43e2-4ebd-bd84-9e729c30256f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:29.185138 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-968145fa-43e2-4ebd-bd84-9e729c30256f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:29.187141 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 772/1552] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:33:29 2022] DELETE /identity/v3/projects/d783018f8c854d42a147376a405f2f7f => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:33:29.192001 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e201f8-250c-403e-9382-e60eefad9ff0 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:29.192835 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e201f8-250c-403e-9382-e60eefad9ff0 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:29.193140 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e201f8-250c-403e-9382-e60eefad9ff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:29.215974 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-57e201f8-250c-403e-9382-e60eefad9ff0 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:33:29.217618 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-57e201f8-250c-403e-9382-e60eefad9ff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:29.255378 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f4c6e5-d3ea-4d3b-83ba-51db0b4a374b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:29.256002 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f4c6e5-d3ea-4d3b-83ba-51db0b4a374b admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:29.256278 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f4c6e5-d3ea-4d3b-83ba-51db0b4a374b admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:29.256650 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f4c6e5-d3ea-4d3b-83ba-51db0b4a374b admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:29.257293 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f4c6e5-d3ea-4d3b-83ba-51db0b4a374b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:29.259555 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f4c6e5-d3ea-4d3b-83ba-51db0b4a374b admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:29.275137 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 781/1553] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:33:29 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 31 12:33:29.279144 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f4c6e5-d3ea-4d3b-83ba-51db0b4a374b admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:29.280949 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-17f4c6e5-d3ea-4d3b-83ba-51db0b4a374b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:29.293350 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f4c6e5-d3ea-4d3b-83ba-51db0b4a374b admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:29.298222 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90278867-8917-4914-a3f0-aaf6399b0300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:29.298875 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90278867-8917-4914-a3f0-aaf6399b0300 admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:29.299690 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90278867-8917-4914-a3f0-aaf6399b0300 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:29.299967 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90278867-8917-4914-a3f0-aaf6399b0300 admin admin] PATH_INFO: `/v3/projects/e32241f95a6146d5b216e8208ea02029/users/9463696528534ac5a11d701cd318a3fa/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:29.307439 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 773/1554] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:33:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 31 12:33:29.308872 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b03cfa-2f35-42d1-8f12-2a9ce47cc16d None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:29.309481 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b03cfa-2f35-42d1-8f12-2a9ce47cc16d None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:29.309481 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b03cfa-2f35-42d1-8f12-2a9ce47cc16d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:29.316440 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90278867-8917-4914-a3f0-aaf6399b0300 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e32241f95a6146d5b216e8208ea02029, user_id=9463696528534ac5a11d701cd318a3fa, role_id=ca1a5695f8bd499aa0b8cedaeecd2cc6)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:29.319260 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90278867-8917-4914-a3f0-aaf6399b0300 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:29.335250 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 782/1555] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:33:29 2022] PUT /identity/v3/projects/e32241f95a6146d5b216e8208ea02029/users/9463696528534ac5a11d701cd318a3fa/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 31 12:33:29.344198 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-75b03cfa-2f35-42d1-8f12-2a9ce47cc16d None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:33:29.344198 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-75b03cfa-2f35-42d1-8f12-2a9ce47cc16d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:29.346527 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa24571-fc7e-4510-831b-0ba99b66cb2c None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:29.346958 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa24571-fc7e-4510-831b-0ba99b66cb2c None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:29.347433 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa24571-fc7e-4510-831b-0ba99b66cb2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:29.393249 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-dfa24571-fc7e-4510-831b-0ba99b66cb2c None None] MFA Rules not processed for user `9463696528534ac5a11d701cd318a3fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:33:29.394411 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-dfa24571-fc7e-4510-831b-0ba99b66cb2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:29.403448 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:33:29.418052 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 774/1556] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:33:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:33:29.421732 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b03cfa-2f35-42d1-8f12-2a9ce47cc16d None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:29.433177 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b03cfa-2f35-42d1-8f12-2a9ce47cc16d None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:29.438102 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cff31b7-8d00-4f95-a014-57787e2a5a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:29.438705 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cff31b7-8d00-4f95-a014-57787e2a5a2a admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:29.438979 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cff31b7-8d00-4f95-a014-57787e2a5a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:29.439276 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cff31b7-8d00-4f95-a014-57787e2a5a2a admin admin] PATH_INFO: `/v3/domains` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:29.440652 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cff31b7-8d00-4f95-a014-57787e2a5a2a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:33:29.440954 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cff31b7-8d00-4f95-a014-57787e2a5a2a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:29.442722 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cff31b7-8d00-4f95-a014-57787e2a5a2a admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:29.459233 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 775/1557] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Oct 31 12:33:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:33:29.461101 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cff31b7-8d00-4f95-a014-57787e2a5a2a admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:29.472309 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cff31b7-8d00-4f95-a014-57787e2a5a2a admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:29.474875 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 783/1558] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:33:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:33:29.478388 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47380203-b146-482c-9736-9272c119b19d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:29.479144 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47380203-b146-482c-9736-9272c119b19d admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:29.479456 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47380203-b146-482c-9736-9272c119b19d admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:29.479727 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47380203-b146-482c-9736-9272c119b19d admin admin] PATH_INFO: `/v3/projects` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:29.480347 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47380203-b146-482c-9736-9272c119b19d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:29.482227 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47380203-b146-482c-9736-9272c119b19d admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:29.490517 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa24571-fc7e-4510-831b-0ba99b66cb2c None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:29.493007 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-dfa24571-fc7e-4510-831b-0ba99b66cb2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:29.505698 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:33:29.519187 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 776/1559] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:33:29 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:33:29.521284 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47380203-b146-482c-9736-9272c119b19d admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:29.558623 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa24571-fc7e-4510-831b-0ba99b66cb2c None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:29.598711 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47380203-b146-482c-9736-9272c119b19d admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:29.604295 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabf8516-2bd0-4077-90c2-feec7022039d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:29.604930 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabf8516-2bd0-4077-90c2-feec7022039d admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:29.605208 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabf8516-2bd0-4077-90c2-feec7022039d admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:29.605477 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabf8516-2bd0-4077-90c2-feec7022039d admin admin] PATH_INFO: `/v3/users` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:29.606101 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabf8516-2bd0-4077-90c2-feec7022039d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:29.608038 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabf8516-2bd0-4077-90c2-feec7022039d admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:29.617623 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af539424-7872-4aea-a82e-45b23386e417 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d6d7ea077d49e8a66c6c6762499702', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:29.617919 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af539424-7872-4aea-a82e-45b23386e417 service cinder] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:29.618166 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af539424-7872-4aea-a82e-45b23386e417 service cinder] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:29.618431 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af539424-7872-4aea-a82e-45b23386e417 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:29.637454 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 777/1560] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:33:29 2022] POST /identity/v3/users => generated 293 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:33:29.642008 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabf8516-2bd0-4077-90c2-feec7022039d admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:29.656062 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabf8516-2bd0-4077-90c2-feec7022039d admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:29.662051 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9277d93f-f6b6-4595-b5f8-fc5586fd5fa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:29.662713 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9277d93f-f6b6-4595-b5f8-fc5586fd5fa0 admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:29.663045 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9277d93f-f6b6-4595-b5f8-fc5586fd5fa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:29.663386 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9277d93f-f6b6-4595-b5f8-fc5586fd5fa0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:29.664201 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9277d93f-f6b6-4595-b5f8-fc5586fd5fa0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:29.666672 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9277d93f-f6b6-4595-b5f8-fc5586fd5fa0 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:29.682668 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 778/1561] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:33:29 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:33:29.686191 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9277d93f-f6b6-4595-b5f8-fc5586fd5fa0 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:29.701585 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9277d93f-f6b6-4595-b5f8-fc5586fd5fa0 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:29.704771 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af539424-7872-4aea-a82e-45b23386e417 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:29.706527 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef90cd66-8ac3-4147-8524-e3e2064c3c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:29.707148 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef90cd66-8ac3-4147-8524-e3e2064c3c69 admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:29.707453 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef90cd66-8ac3-4147-8524-e3e2064c3c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:29.707725 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef90cd66-8ac3-4147-8524-e3e2064c3c69 admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:29.708256 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af539424-7872-4aea-a82e-45b23386e417 service cinder] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:29.708413 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef90cd66-8ac3-4147-8524-e3e2064c3c69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:29.710024 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef90cd66-8ac3-4147-8524-e3e2064c3c69 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:29.719457 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 784/1562] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:33:29 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 31 12:33:29.724035 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 779/1563] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:33:29 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:33:29.728195 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af539424-7872-4aea-a82e-45b23386e417 service cinder] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:29.740696 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af539424-7872-4aea-a82e-45b23386e417 service cinder] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:29.745730 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba86debe-8cc6-418b-a31d-292014db5085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:29.746628 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba86debe-8cc6-418b-a31d-292014db5085 admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:29.747094 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba86debe-8cc6-418b-a31d-292014db5085 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:29.747961 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba86debe-8cc6-418b-a31d-292014db5085 admin admin] PATH_INFO: `/v3/projects/054100cd472a42679edc5b2b02a069a6/users/8df2416ea8e14ba68fd4b7ad47ea231a/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:29.778827 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba86debe-8cc6-418b-a31d-292014db5085 admin admin] RBAC: Authorizing `identity:create_grant(project_id=054100cd472a42679edc5b2b02a069a6, user_id=8df2416ea8e14ba68fd4b7ad47ea231a, role_id=ca1a5695f8bd499aa0b8cedaeecd2cc6)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:29.782115 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba86debe-8cc6-418b-a31d-292014db5085 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:29.807795 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 785/1564] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:33:29 2022] PUT /identity/v3/projects/054100cd472a42679edc5b2b02a069a6/users/8df2416ea8e14ba68fd4b7ad47ea231a/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:33:29.816361 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080838e2-8ac1-48e6-bca3-6a3de0bbe025 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:29.816647 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080838e2-8ac1-48e6-bca3-6a3de0bbe025 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:29.816925 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080838e2-8ac1-48e6-bca3-6a3de0bbe025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:29.864609 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-080838e2-8ac1-48e6-bca3-6a3de0bbe025 None None] MFA Rules not processed for user `8df2416ea8e14ba68fd4b7ad47ea231a`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:33:29.865851 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-080838e2-8ac1-48e6-bca3-6a3de0bbe025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:29.935589 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 780/1565] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:33:29 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:33:29.949349 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba86debe-8cc6-418b-a31d-292014db5085 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:29.951518 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-ba86debe-8cc6-418b-a31d-292014db5085 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:30.036280 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba86debe-8cc6-418b-a31d-292014db5085 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:30.040283 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0b52af-2698-48de-8dbe-00f934fb84ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d6d7ea077d49e8a66c6c6762499702', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:30.040904 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0b52af-2698-48de-8dbe-00f934fb84ab service cinder] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:30.041185 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0b52af-2698-48de-8dbe-00f934fb84ab service cinder] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:30.041459 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0b52af-2698-48de-8dbe-00f934fb84ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:30.051387 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0b52af-2698-48de-8dbe-00f934fb84ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:30.053113 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0b52af-2698-48de-8dbe-00f934fb84ab service cinder] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:30.057215 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:33:30.069438 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 786/1566] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:33:29 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:33:39.932730 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=88,keystone:INSERT=4 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:33:40.075686 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:33:40.886674 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080838e2-8ac1-48e6-bca3-6a3de0bbe025 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:40.888235 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-080838e2-8ac1-48e6-bca3-6a3de0bbe025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:40.958849 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080838e2-8ac1-48e6-bca3-6a3de0bbe025 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:40.964306 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fadf6e-614a-4c54-a3f4-51ddacfa7a08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:40.965128 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fadf6e-614a-4c54-a3f4-51ddacfa7a08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:40.965531 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fadf6e-614a-4c54-a3f4-51ddacfa7a08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:40.965909 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fadf6e-614a-4c54-a3f4-51ddacfa7a08 admin admin] PATH_INFO: `/v3/users/5108166ba8d3427c8f7e7fe23fa61881` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:40.967610 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fadf6e-614a-4c54-a3f4-51ddacfa7a08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5108166ba8d3427c8f7e7fe23fa61881)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:40.969888 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fadf6e-614a-4c54-a3f4-51ddacfa7a08 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:41.019943 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-90fadf6e-614a-4c54-a3f4-51ddacfa7a08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5108166ba8d3427c8f7e7fe23fa61881'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:41.028783 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-90fadf6e-614a-4c54-a3f4-51ddacfa7a08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5108166ba8d3427c8f7e7fe23fa61881'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:41.030061 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-90fadf6e-614a-4c54-a3f4-51ddacfa7a08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5108166ba8d3427c8f7e7fe23fa61881'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:41.043838 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-90fadf6e-614a-4c54-a3f4-51ddacfa7a08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5108166ba8d3427c8f7e7fe23fa61881'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:41.051504 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-90fadf6e-614a-4c54-a3f4-51ddacfa7a08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5108166ba8d3427c8f7e7fe23fa61881'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:41.057453 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 781/1567] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:33:40 2022] DELETE /identity/v3/users/5108166ba8d3427c8f7e7fe23fa61881 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:33:41.061994 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2864ee10-401f-4cb0-8015-6db30569900b None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:41.062274 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2864ee10-401f-4cb0-8015-6db30569900b None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:41.062546 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2864ee10-401f-4cb0-8015-6db30569900b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:41.082259 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-2864ee10-401f-4cb0-8015-6db30569900b None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:33:41.083502 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-2864ee10-401f-4cb0-8015-6db30569900b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:41.151305 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 787/1568] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:33:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:33:41.168417 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fadf6e-614a-4c54-a3f4-51ddacfa7a08 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:41.170231 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-90fadf6e-614a-4c54-a3f4-51ddacfa7a08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:41.173814 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6084cb5-da09-4575-a7ea-77036ec446f2 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:41.175786 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6084cb5-da09-4575-a7ea-77036ec446f2 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:41.175982 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6084cb5-da09-4575-a7ea-77036ec446f2 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:41.181741 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 788/1569] 10.222.0.15 () {54 vars in 921 bytes} [Mon Oct 31 12:33:41 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:33:41.236357 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bbdc60-be88-4b3c-93b6-0fb558d5f64f None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:41.237938 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bbdc60-be88-4b3c-93b6-0fb558d5f64f None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:41.238668 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bbdc60-be88-4b3c-93b6-0fb558d5f64f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:41.264517 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fadf6e-614a-4c54-a3f4-51ddacfa7a08 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:41.269247 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6968d00f-3c19-420d-95c7-e390293848ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:41.269874 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6968d00f-3c19-420d-95c7-e390293848ae service neutron] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:41.270161 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6968d00f-3c19-420d-95c7-e390293848ae service neutron] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:41.270795 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6968d00f-3c19-420d-95c7-e390293848ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:41.284672 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-31bbdc60-be88-4b3c-93b6-0fb558d5f64f None None] MFA Rules not processed for user `99889b1dab0b471d9c77887cffefc652`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:33:41.285489 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6968d00f-3c19-420d-95c7-e390293848ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:41.287417 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6968d00f-3c19-420d-95c7-e390293848ae service neutron] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:41.288601 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-31bbdc60-be88-4b3c-93b6-0fb558d5f64f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:41.298627 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 782/1570] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:33:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:33:41.404588 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 789/1571] 10.222.0.15 () {58 vars in 1018 bytes} [Mon Oct 31 12:33:41 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 31 12:33:41.445469 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6968d00f-3c19-420d-95c7-e390293848ae service neutron] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:41.448470 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-6968d00f-3c19-420d-95c7-e390293848ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:41.481747 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6968d00f-3c19-420d-95c7-e390293848ae service neutron] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:41.494492 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560c5d26-9ada-476b-8969-86ff7ba2e021 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99889b1dab0b471d9c77887cffefc652', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:41.495902 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560c5d26-9ada-476b-8969-86ff7ba2e021 service nova] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:41.496261 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560c5d26-9ada-476b-8969-86ff7ba2e021 service nova] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:41.497385 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560c5d26-9ada-476b-8969-86ff7ba2e021 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:41.509777 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560c5d26-9ada-476b-8969-86ff7ba2e021 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:41.512343 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560c5d26-9ada-476b-8969-86ff7ba2e021 service nova] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:41.542145 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 783/1572] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:33:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:33:42.385649 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bbdc60-be88-4b3c-93b6-0fb558d5f64f None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:42.388309 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-31bbdc60-be88-4b3c-93b6-0fb558d5f64f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:42.410720 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bbdc60-be88-4b3c-93b6-0fb558d5f64f None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:42.419827 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bd45f2-3a0f-4ca1-864a-7256474f5f65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:42.421572 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bd45f2-3a0f-4ca1-864a-7256474f5f65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:42.422069 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bd45f2-3a0f-4ca1-864a-7256474f5f65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:42.422541 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bd45f2-3a0f-4ca1-864a-7256474f5f65 admin admin] PATH_INFO: `/v3/projects/ea42f548e2df4e9f8a8e72e4243950d1` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:42.424730 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bd45f2-3a0f-4ca1-864a-7256474f5f65 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea42f548e2df4e9f8a8e72e4243950d1)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:42.427609 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bd45f2-3a0f-4ca1-864a-7256474f5f65 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:42.509582 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-d4bd45f2-3a0f-4ca1-864a-7256474f5f65 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea42f548e2df4e9f8a8e72e4243950d1', 'request_id': 'req-d4bd45f2-3a0f-4ca1-864a-7256474f5f65'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:42.518414 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-d4bd45f2-3a0f-4ca1-864a-7256474f5f65 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea42f548e2df4e9f8a8e72e4243950d1', 'request_id': 'req-d4bd45f2-3a0f-4ca1-864a-7256474f5f65'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:42.540130 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-d4bd45f2-3a0f-4ca1-864a-7256474f5f65 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea42f548e2df4e9f8a8e72e4243950d1', 'request_id': 'req-d4bd45f2-3a0f-4ca1-864a-7256474f5f65'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:42.581510 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-d4bd45f2-3a0f-4ca1-864a-7256474f5f65 admin admin] The token cache is being invalidate because project ea42f548e2df4e9f8a8e72e4243950d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:33:42.582046 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-d4bd45f2-3a0f-4ca1-864a-7256474f5f65 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:42.582986 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-d4bd45f2-3a0f-4ca1-864a-7256474f5f65 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:42.585101 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 790/1573] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:33:42 2022] DELETE /identity/v3/projects/ea42f548e2df4e9f8a8e72e4243950d1 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:33:42.598524 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560c5d26-9ada-476b-8969-86ff7ba2e021 service nova] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:42.618951 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560c5d26-9ada-476b-8969-86ff7ba2e021 service nova] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:42.626040 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:33:42.727737 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8d8343-e564-47aa-a8d9-9787474043df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:42.728389 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8d8343-e564-47aa-a8d9-9787474043df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:42.728685 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8d8343-e564-47aa-a8d9-9787474043df admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:42.728942 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8d8343-e564-47aa-a8d9-9787474043df admin admin] PATH_INFO: `/v3/users/ee460f0372324718a523c7852a85358c` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:42.730097 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8d8343-e564-47aa-a8d9-9787474043df admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee460f0372324718a523c7852a85358c)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:42.732246 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8d8343-e564-47aa-a8d9-9787474043df admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:42.829359 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bd45f2-3a0f-4ca1-864a-7256474f5f65 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:42.833100 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-3c8d8343-e564-47aa-a8d9-9787474043df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee460f0372324718a523c7852a85358c'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:42.833827 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-d4bd45f2-3a0f-4ca1-864a-7256474f5f65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:42.848503 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-3c8d8343-e564-47aa-a8d9-9787474043df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee460f0372324718a523c7852a85358c'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:42.849752 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-3c8d8343-e564-47aa-a8d9-9787474043df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee460f0372324718a523c7852a85358c'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:42.852084 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bd45f2-3a0f-4ca1-864a-7256474f5f65 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:42.864433 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ded519c-8ea3-47f3-83f3-89946a7ca46f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:42.865118 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ded519c-8ea3-47f3-83f3-89946a7ca46f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:42.865476 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ded519c-8ea3-47f3-83f3-89946a7ca46f admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:42.865823 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ded519c-8ea3-47f3-83f3-89946a7ca46f admin admin] PATH_INFO: `/v3/users/8df2416ea8e14ba68fd4b7ad47ea231a` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:42.867052 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ded519c-8ea3-47f3-83f3-89946a7ca46f admin admin] RBAC: Authorizing `identity:delete_user(user_id=8df2416ea8e14ba68fd4b7ad47ea231a)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:42.869592 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ded519c-8ea3-47f3-83f3-89946a7ca46f admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:42.881461 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-3c8d8343-e564-47aa-a8d9-9787474043df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee460f0372324718a523c7852a85358c'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:42.883793 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:33:42.908876 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-3c8d8343-e564-47aa-a8d9-9787474043df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee460f0372324718a523c7852a85358c'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:42.918189 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 784/1574] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:33:42 2022] DELETE /identity/v3/users/ee460f0372324718a523c7852a85358c => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:33:42.937640 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-0ded519c-8ea3-47f3-83f3-89946a7ca46f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8df2416ea8e14ba68fd4b7ad47ea231a'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:42.947604 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-0ded519c-8ea3-47f3-83f3-89946a7ca46f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8df2416ea8e14ba68fd4b7ad47ea231a'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:42.949191 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-0ded519c-8ea3-47f3-83f3-89946a7ca46f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8df2416ea8e14ba68fd4b7ad47ea231a'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:42.949935 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-0ded519c-8ea3-47f3-83f3-89946a7ca46f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8df2416ea8e14ba68fd4b7ad47ea231a'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:42.964740 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-0ded519c-8ea3-47f3-83f3-89946a7ca46f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8df2416ea8e14ba68fd4b7ad47ea231a'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:42.980845 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 791/1575] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:33:42 2022] DELETE /identity/v3/users/8df2416ea8e14ba68fd4b7ad47ea231a => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:33:42.985307 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7173a1-6443-4255-b578-a842eb4dee0f None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:42.986212 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7173a1-6443-4255-b578-a842eb4dee0f None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:42.986509 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7173a1-6443-4255-b578-a842eb4dee0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:43.008244 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-6d7173a1-6443-4255-b578-a842eb4dee0f None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:33:43.009463 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-6d7173a1-6443-4255-b578-a842eb4dee0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:43.106945 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 785/1576] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:33:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:33:43.122164 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ded519c-8ea3-47f3-83f3-89946a7ca46f admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:43.124860 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-0ded519c-8ea3-47f3-83f3-89946a7ca46f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:43.194719 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ded519c-8ea3-47f3-83f3-89946a7ca46f admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:43.199423 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbbcfd1-ec96-42dd-bcb7-842975668a3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:43.200109 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbbcfd1-ec96-42dd-bcb7-842975668a3d service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:43.200447 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbbcfd1-ec96-42dd-bcb7-842975668a3d service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:43.200996 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbbcfd1-ec96-42dd-bcb7-842975668a3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:43.210628 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbbcfd1-ec96-42dd-bcb7-842975668a3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:43.212377 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbbcfd1-ec96-42dd-bcb7-842975668a3d service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:43.230787 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 792/1577] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:33:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:33:43.438502 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7173a1-6443-4255-b578-a842eb4dee0f None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:43.440003 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-6d7173a1-6443-4255-b578-a842eb4dee0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:43.453179 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7173a1-6443-4255-b578-a842eb4dee0f None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:43.458700 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eda87a-bf18-43c4-82df-e34c398809f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:43.459374 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eda87a-bf18-43c4-82df-e34c398809f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:43.459651 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eda87a-bf18-43c4-82df-e34c398809f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:43.459923 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eda87a-bf18-43c4-82df-e34c398809f6 admin admin] PATH_INFO: `/v3/projects/e707c2b2e76f4cdf8f41b4e9863c2e9e` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:43.461737 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12eda87a-bf18-43c4-82df-e34c398809f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e707c2b2e76f4cdf8f41b4e9863c2e9e)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:43.463478 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12eda87a-bf18-43c4-82df-e34c398809f6 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:43.504093 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:33:43.533207 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-12eda87a-bf18-43c4-82df-e34c398809f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e707c2b2e76f4cdf8f41b4e9863c2e9e', 'request_id': 'req-12eda87a-bf18-43c4-82df-e34c398809f6'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:43.541860 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-12eda87a-bf18-43c4-82df-e34c398809f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e707c2b2e76f4cdf8f41b4e9863c2e9e', 'request_id': 'req-12eda87a-bf18-43c4-82df-e34c398809f6'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:43.552229 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-12eda87a-bf18-43c4-82df-e34c398809f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e707c2b2e76f4cdf8f41b4e9863c2e9e', 'request_id': 'req-12eda87a-bf18-43c4-82df-e34c398809f6'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:43.563971 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-12eda87a-bf18-43c4-82df-e34c398809f6 admin admin] The token cache is being invalidate because project e707c2b2e76f4cdf8f41b4e9863c2e9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:33:43.564318 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-12eda87a-bf18-43c4-82df-e34c398809f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:43.565300 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-12eda87a-bf18-43c4-82df-e34c398809f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:43.567257 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 786/1578] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:33:43 2022] DELETE /identity/v3/projects/e707c2b2e76f4cdf8f41b4e9863c2e9e => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:33:43.571814 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7987a5b4-30da-4a5f-9385-6932caaf5393 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:43.572269 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7987a5b4-30da-4a5f-9385-6932caaf5393 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:43.572373 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7987a5b4-30da-4a5f-9385-6932caaf5393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:43.598751 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-7987a5b4-30da-4a5f-9385-6932caaf5393 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:33:43.600532 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-7987a5b4-30da-4a5f-9385-6932caaf5393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:43.707868 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 793/1579] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:33:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:33:43.714872 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c82c279-a1bb-4ff9-9651-c6390554a1d3 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:43.715366 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c82c279-a1bb-4ff9-9651-c6390554a1d3 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:43.715440 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c82c279-a1bb-4ff9-9651-c6390554a1d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:43.738627 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-8c82c279-a1bb-4ff9-9651-c6390554a1d3 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:33:43.739960 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-8c82c279-a1bb-4ff9-9651-c6390554a1d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:43.751194 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 787/1580] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:33:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:33:43.756821 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7987a5b4-30da-4a5f-9385-6932caaf5393 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:43.768588 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7987a5b4-30da-4a5f-9385-6932caaf5393 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:43.775884 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd05474b-da73-4e91-be1f-08d9f2e1383d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:43.776476 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd05474b-da73-4e91-be1f-08d9f2e1383d admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:43.776890 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd05474b-da73-4e91-be1f-08d9f2e1383d admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:43.777083 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd05474b-da73-4e91-be1f-08d9f2e1383d admin admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:43.777706 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd05474b-da73-4e91-be1f-08d9f2e1383d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:33:43.778065 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd05474b-da73-4e91-be1f-08d9f2e1383d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:43.780153 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd05474b-da73-4e91-be1f-08d9f2e1383d admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:43.780585 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c82c279-a1bb-4ff9-9651-c6390554a1d3 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:43.781971 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-8c82c279-a1bb-4ff9-9651-c6390554a1d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:43.795210 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 794/1581] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Oct 31 12:33:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:33:43.808353 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd05474b-da73-4e91-be1f-08d9f2e1383d admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:43.811232 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c82c279-a1bb-4ff9-9651-c6390554a1d3 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:43.834966 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf24a38-19b7-4f08-9b2c-72b268a97be0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:43.835659 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf24a38-19b7-4f08-9b2c-72b268a97be0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:43.835930 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf24a38-19b7-4f08-9b2c-72b268a97be0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:43.836270 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf24a38-19b7-4f08-9b2c-72b268a97be0 admin admin] PATH_INFO: `/v3/projects/054100cd472a42679edc5b2b02a069a6` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:43.848682 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd05474b-da73-4e91-be1f-08d9f2e1383d admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:43.849054 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf24a38-19b7-4f08-9b2c-72b268a97be0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=054100cd472a42679edc5b2b02a069a6)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:43.853022 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf24a38-19b7-4f08-9b2c-72b268a97be0 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:43.855321 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b2e9c6-9c8a-4456-a39c-edac6e293a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:43.856048 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b2e9c6-9c8a-4456-a39c-edac6e293a73 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:43.856213 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b2e9c6-9c8a-4456-a39c-edac6e293a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:43.856489 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b2e9c6-9c8a-4456-a39c-edac6e293a73 admin admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:43.857145 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b2e9c6-9c8a-4456-a39c-edac6e293a73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:43.859031 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b2e9c6-9c8a-4456-a39c-edac6e293a73 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:43.876830 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:33:43.895438 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 795/1582] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:33:43 2022] POST /identity/v3/projects => generated 387 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:33:43.896875 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b2e9c6-9c8a-4456-a39c-edac6e293a73 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:43.930752 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-9cf24a38-19b7-4f08-9b2c-72b268a97be0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '054100cd472a42679edc5b2b02a069a6', 'request_id': 'req-9cf24a38-19b7-4f08-9b2c-72b268a97be0'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:43.939199 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-9cf24a38-19b7-4f08-9b2c-72b268a97be0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '054100cd472a42679edc5b2b02a069a6', 'request_id': 'req-9cf24a38-19b7-4f08-9b2c-72b268a97be0'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:43.950176 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-9cf24a38-19b7-4f08-9b2c-72b268a97be0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '054100cd472a42679edc5b2b02a069a6', 'request_id': 'req-9cf24a38-19b7-4f08-9b2c-72b268a97be0'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:43.967954 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-9cf24a38-19b7-4f08-9b2c-72b268a97be0 admin admin] The token cache is being invalidate because project 054100cd472a42679edc5b2b02a069a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:33:43.968339 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-9cf24a38-19b7-4f08-9b2c-72b268a97be0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:43.968996 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-9cf24a38-19b7-4f08-9b2c-72b268a97be0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:43.974451 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 788/1583] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:33:43 2022] DELETE /identity/v3/projects/054100cd472a42679edc5b2b02a069a6 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 31 12:33:43.974589 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b2e9c6-9c8a-4456-a39c-edac6e293a73 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:43.975868 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d124c4-ec9b-4f0c-91f0-c6b0f86db71b None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:43.976217 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d124c4-ec9b-4f0c-91f0-c6b0f86db71b None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:43.976536 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d124c4-ec9b-4f0c-91f0-c6b0f86db71b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:43.998549 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-86d124c4-ec9b-4f0c-91f0-c6b0f86db71b None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:33:43.999797 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-86d124c4-ec9b-4f0c-91f0-c6b0f86db71b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:44.044035 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e67559-e161-45f9-bc84-13b41ec18b81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:44.044754 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e67559-e161-45f9-bc84-13b41ec18b81 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:44.045464 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e67559-e161-45f9-bc84-13b41ec18b81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:44.045741 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e67559-e161-45f9-bc84-13b41ec18b81 admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:44.046400 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e67559-e161-45f9-bc84-13b41ec18b81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:44.049285 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e67559-e161-45f9-bc84-13b41ec18b81 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:44.067027 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 789/1584] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:33:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:33:44.071492 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 796/1585] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:33:43 2022] POST /identity/v3/users => generated 294 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 31 12:33:44.075565 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d124c4-ec9b-4f0c-91f0-c6b0f86db71b None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:44.076929 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-86d124c4-ec9b-4f0c-91f0-c6b0f86db71b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:44.078440 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6bf078-fce1-49c6-85fa-0364cf7fdf4b None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:44.078730 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6bf078-fce1-49c6-85fa-0364cf7fdf4b None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:44.079005 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6bf078-fce1-49c6-85fa-0364cf7fdf4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:44.089438 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d124c4-ec9b-4f0c-91f0-c6b0f86db71b None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:44.094006 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510f434a-c1a8-4f60-84f5-64a596feda60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:44.094606 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510f434a-c1a8-4f60-84f5-64a596feda60 admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:44.094884 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510f434a-c1a8-4f60-84f5-64a596feda60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:44.095155 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510f434a-c1a8-4f60-84f5-64a596feda60 admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:44.095819 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510f434a-c1a8-4f60-84f5-64a596feda60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:44.098223 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510f434a-c1a8-4f60-84f5-64a596feda60 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:44.100001 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-0c6bf078-fce1-49c6-85fa-0364cf7fdf4b None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:33:44.101189 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-0c6bf078-fce1-49c6-85fa-0364cf7fdf4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:44.109043 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 797/1586] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:33:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 31 12:33:44.109424 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 790/1587] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:33:44 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 31 12:33:44.112979 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6bf078-fce1-49c6-85fa-0364cf7fdf4b None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:44.115333 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510f434a-c1a8-4f60-84f5-64a596feda60 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:44.122856 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6bf078-fce1-49c6-85fa-0364cf7fdf4b None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:44.125828 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510f434a-c1a8-4f60-84f5-64a596feda60 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:44.128259 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e408d1b7-8887-427a-ae95-816d293c3b7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:44.128487 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e408d1b7-8887-427a-ae95-816d293c3b7d admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:44.128647 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e408d1b7-8887-427a-ae95-816d293c3b7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:44.128986 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e408d1b7-8887-427a-ae95-816d293c3b7d admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:44.129663 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e408d1b7-8887-427a-ae95-816d293c3b7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:44.130767 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a0f446-6681-44cb-a885-995a4512bbb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:44.131402 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a0f446-6681-44cb-a885-995a4512bbb4 admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:44.131516 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e408d1b7-8887-427a-ae95-816d293c3b7d admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:44.131677 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a0f446-6681-44cb-a885-995a4512bbb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:44.132182 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a0f446-6681-44cb-a885-995a4512bbb4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:44.133010 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a0f446-6681-44cb-a885-995a4512bbb4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:33:44.133291 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a0f446-6681-44cb-a885-995a4512bbb4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:44.135635 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a0f446-6681-44cb-a885-995a4512bbb4 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:44.144252 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 798/1588] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:33:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:33:44.148608 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e408d1b7-8887-427a-ae95-816d293c3b7d admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:44.150144 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 791/1589] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Oct 31 12:33:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 31 12:33:44.153974 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a0f446-6681-44cb-a885-995a4512bbb4 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:44.160725 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e408d1b7-8887-427a-ae95-816d293c3b7d admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:44.161641 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:33:44.165615 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a0f446-6681-44cb-a885-995a4512bbb4 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:44.167217 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0673ea-21b8-4567-8b90-b0238cbd5a47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:44.168041 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0673ea-21b8-4567-8b90-b0238cbd5a47 admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:44.168203 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0673ea-21b8-4567-8b90-b0238cbd5a47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:44.168456 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0673ea-21b8-4567-8b90-b0238cbd5a47 admin admin] PATH_INFO: `/v3/projects/76a044019c324ceb898e237b1aa92330/users/1e871325d6ca43398cf883cfe53914d1/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:44.172995 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6207a98a-499b-420a-8eae-8f744521fd27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:44.174372 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6207a98a-499b-420a-8eae-8f744521fd27 admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:44.175405 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6207a98a-499b-420a-8eae-8f744521fd27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:44.176219 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6207a98a-499b-420a-8eae-8f744521fd27 admin admin] PATH_INFO: `/v3/projects` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:44.177470 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6207a98a-499b-420a-8eae-8f744521fd27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:44.181082 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6207a98a-499b-420a-8eae-8f744521fd27 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:44.184443 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0673ea-21b8-4567-8b90-b0238cbd5a47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76a044019c324ceb898e237b1aa92330, user_id=1e871325d6ca43398cf883cfe53914d1, role_id=ca1a5695f8bd499aa0b8cedaeecd2cc6)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:44.187336 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0673ea-21b8-4567-8b90-b0238cbd5a47 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:44.202686 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 799/1590] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:33:44 2022] PUT /identity/v3/projects/76a044019c324ceb898e237b1aa92330/users/1e871325d6ca43398cf883cfe53914d1/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:33:44.214643 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac95c58-a68d-4b59-ba4f-3bd3d190d44c None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:44.214933 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac95c58-a68d-4b59-ba4f-3bd3d190d44c None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:44.215244 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac95c58-a68d-4b59-ba4f-3bd3d190d44c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:44.221838 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 792/1591] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:33:44 2022] POST /identity/v3/projects => generated 361 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 31 12:33:44.226463 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6207a98a-499b-420a-8eae-8f744521fd27 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:44.253096 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-2ac95c58-a68d-4b59-ba4f-3bd3d190d44c None None] MFA Rules not processed for user `1e871325d6ca43398cf883cfe53914d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:33:44.254346 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-2ac95c58-a68d-4b59-ba4f-3bd3d190d44c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:44.322301 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:33:44.325242 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6207a98a-499b-420a-8eae-8f744521fd27 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:44.335030 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da5c93a-d70a-4ea6-8190-67fdaf7ff4dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:44.336459 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da5c93a-d70a-4ea6-8190-67fdaf7ff4dc admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:44.337135 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da5c93a-d70a-4ea6-8190-67fdaf7ff4dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:44.338152 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da5c93a-d70a-4ea6-8190-67fdaf7ff4dc admin admin] PATH_INFO: `/v3/users` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:44.339727 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da5c93a-d70a-4ea6-8190-67fdaf7ff4dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:44.344087 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da5c93a-d70a-4ea6-8190-67fdaf7ff4dc admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:44.357145 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 800/1592] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:33:44 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:33:44.361273 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac95c58-a68d-4b59-ba4f-3bd3d190d44c None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:44.374968 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac95c58-a68d-4b59-ba4f-3bd3d190d44c None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:44.380658 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aed11f-7ea9-468e-8ee1-3c14c423b75f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:44.381371 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aed11f-7ea9-468e-8ee1-3c14c423b75f admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:44.381655 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aed11f-7ea9-468e-8ee1-3c14c423b75f admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:44.382070 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aed11f-7ea9-468e-8ee1-3c14c423b75f admin admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:44.385454 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 793/1593] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:33:44 2022] POST /identity/v3/users => generated 281 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 31 12:33:44.385584 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66aed11f-7ea9-468e-8ee1-3c14c423b75f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:44.385584 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66aed11f-7ea9-468e-8ee1-3c14c423b75f admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:44.387494 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da5c93a-d70a-4ea6-8190-67fdaf7ff4dc admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:44.404053 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da5c93a-d70a-4ea6-8190-67fdaf7ff4dc admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:44.410846 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6882825-cd6e-4643-8cea-191b47308e9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:44.411514 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6882825-cd6e-4643-8cea-191b47308e9f admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:44.411791 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6882825-cd6e-4643-8cea-191b47308e9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:44.412056 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6882825-cd6e-4643-8cea-191b47308e9f admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:44.412679 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6882825-cd6e-4643-8cea-191b47308e9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:44.414262 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6882825-cd6e-4643-8cea-191b47308e9f admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:44.417407 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 801/1594] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:33:44 2022] POST /identity/v3/projects => generated 387 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 31 12:33:44.421325 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aed11f-7ea9-468e-8ee1-3c14c423b75f admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:44.432164 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 794/1595] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:33:44 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:33:44.433578 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6882825-cd6e-4643-8cea-191b47308e9f admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:44.444390 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6882825-cd6e-4643-8cea-191b47308e9f admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:44.492068 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aed11f-7ea9-468e-8ee1-3c14c423b75f admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:44.497848 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aaf80f-6194-4383-82d8-c775a4c46d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:44.498462 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aaf80f-6194-4383-82d8-c775a4c46d50 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:44.499010 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aaf80f-6194-4383-82d8-c775a4c46d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:44.499010 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aaf80f-6194-4383-82d8-c775a4c46d50 admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:44.499653 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aaf80f-6194-4383-82d8-c775a4c46d50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:44.501558 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aaf80f-6194-4383-82d8-c775a4c46d50 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:44.505382 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057b7440-12ce-4188-a3f1-02eed8ddd6ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:44.506057 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b7440-12ce-4188-a3f1-02eed8ddd6ff admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:44.506402 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b7440-12ce-4188-a3f1-02eed8ddd6ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:44.506753 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b7440-12ce-4188-a3f1-02eed8ddd6ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:44.507486 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057b7440-12ce-4188-a3f1-02eed8ddd6ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:44.509107 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057b7440-12ce-4188-a3f1-02eed8ddd6ff admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:44.521926 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 795/1596] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:33:44 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 31 12:33:44.524638 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057b7440-12ce-4188-a3f1-02eed8ddd6ff admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:44.531441 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 802/1597] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:33:44 2022] POST /identity/v3/users => generated 300 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:33:44.533282 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aaf80f-6194-4383-82d8-c775a4c46d50 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:44.536778 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057b7440-12ce-4188-a3f1-02eed8ddd6ff admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:44.541395 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0b809b-f45b-46d6-9c6c-4f21431bd7e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:44.542053 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0b809b-f45b-46d6-9c6c-4f21431bd7e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:44.542330 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0b809b-f45b-46d6-9c6c-4f21431bd7e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:44.542618 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0b809b-f45b-46d6-9c6c-4f21431bd7e3 admin admin] PATH_INFO: `/v3/projects/7167996586f14b54aeb833c5615e290f/users/d1a39c9218d744c89085e65cb3e07819/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:44.544395 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aaf80f-6194-4383-82d8-c775a4c46d50 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:44.551525 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94e8e47-317f-4c3a-91f2-2f144410eca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:44.552221 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94e8e47-317f-4c3a-91f2-2f144410eca4 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:44.552794 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94e8e47-317f-4c3a-91f2-2f144410eca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:44.553153 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94e8e47-317f-4c3a-91f2-2f144410eca4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:44.553877 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94e8e47-317f-4c3a-91f2-2f144410eca4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:44.555838 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94e8e47-317f-4c3a-91f2-2f144410eca4 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:44.557340 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0b809b-f45b-46d6-9c6c-4f21431bd7e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7167996586f14b54aeb833c5615e290f, user_id=d1a39c9218d744c89085e65cb3e07819, role_id=ca1a5695f8bd499aa0b8cedaeecd2cc6)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:44.560000 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0b809b-f45b-46d6-9c6c-4f21431bd7e3 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:44.568654 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 803/1598] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:33:44 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:33:44.572827 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94e8e47-317f-4c3a-91f2-2f144410eca4 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:44.577156 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 796/1599] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:33:44 2022] PUT /identity/v3/projects/7167996586f14b54aeb833c5615e290f/users/d1a39c9218d744c89085e65cb3e07819/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 31 12:33:44.585944 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94e8e47-317f-4c3a-91f2-2f144410eca4 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:44.587083 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4adcf79-aa85-4e47-96b6-7fbacef724c9 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:44.587395 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4adcf79-aa85-4e47-96b6-7fbacef724c9 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:44.587665 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4adcf79-aa85-4e47-96b6-7fbacef724c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:44.617533 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-b4adcf79-aa85-4e47-96b6-7fbacef724c9 None None] MFA Rules not processed for user `d1a39c9218d744c89085e65cb3e07819`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:33:44.618880 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-b4adcf79-aa85-4e47-96b6-7fbacef724c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:44.640890 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:33:44.648565 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4052887-631a-45e8-9da7-09036905568c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:44.649194 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4052887-631a-45e8-9da7-09036905568c admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:44.649471 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4052887-631a-45e8-9da7-09036905568c admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:44.649742 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4052887-631a-45e8-9da7-09036905568c admin admin] PATH_INFO: `/v3/projects/989774f7ec3d47edbb69cd843407272f/users/27fe920f2bee4c4f8133b762e7b9e8dd/roles/176c07b4fe3b4537bdfeb4be21b379c5` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:44.667026 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4052887-631a-45e8-9da7-09036905568c admin admin] RBAC: Authorizing `identity:create_grant(project_id=989774f7ec3d47edbb69cd843407272f, user_id=27fe920f2bee4c4f8133b762e7b9e8dd, role_id=176c07b4fe3b4537bdfeb4be21b379c5)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:44.669837 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4052887-631a-45e8-9da7-09036905568c admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:44.686809 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 804/1600] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:33:44 2022] PUT /identity/v3/projects/989774f7ec3d47edbb69cd843407272f/users/27fe920f2bee4c4f8133b762e7b9e8dd/roles/176c07b4fe3b4537bdfeb4be21b379c5 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:33:44.691566 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4052887-631a-45e8-9da7-09036905568c admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:44.695631 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 797/1601] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:33:44 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 31 12:33:44.714697 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:33:44.732273 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4adcf79-aa85-4e47-96b6-7fbacef724c9 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:44.733663 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-b4adcf79-aa85-4e47-96b6-7fbacef724c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:44.758281 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4052887-631a-45e8-9da7-09036905568c admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:44.763117 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f67e22-a474-4e02-ad7f-ffdd6c252eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:44.763749 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f67e22-a474-4e02-ad7f-ffdd6c252eb9 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:44.764029 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f67e22-a474-4e02-ad7f-ffdd6c252eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:44.764295 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f67e22-a474-4e02-ad7f-ffdd6c252eb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:44.764923 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f67e22-a474-4e02-ad7f-ffdd6c252eb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:44.766523 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f67e22-a474-4e02-ad7f-ffdd6c252eb9 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:44.776674 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 805/1602] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:33:44 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 31 12:33:44.780741 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f67e22-a474-4e02-ad7f-ffdd6c252eb9 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:44.790855 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f67e22-a474-4e02-ad7f-ffdd6c252eb9 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:44.794366 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4adcf79-aa85-4e47-96b6-7fbacef724c9 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:44.795646 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f702fe52-c812-494d-87cb-65c54ad235c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:44.796302 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f702fe52-c812-494d-87cb-65c54ad235c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:44.796579 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f702fe52-c812-494d-87cb-65c54ad235c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:44.796847 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f702fe52-c812-494d-87cb-65c54ad235c3 admin admin] PATH_INFO: `/v3/projects/989774f7ec3d47edbb69cd843407272f/users/27fe920f2bee4c4f8133b762e7b9e8dd/roles/176c07b4fe3b4537bdfeb4be21b379c5` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:44.798737 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f702fe52-c812-494d-87cb-65c54ad235c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=989774f7ec3d47edbb69cd843407272f, user_id=27fe920f2bee4c4f8133b762e7b9e8dd, role_id=176c07b4fe3b4537bdfeb4be21b379c5)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:44.802220 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f702fe52-c812-494d-87cb-65c54ad235c3 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:44.812337 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 806/1603] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:33:44 2022] PUT /identity/v3/projects/989774f7ec3d47edbb69cd843407272f/users/27fe920f2bee4c4f8133b762e7b9e8dd/roles/176c07b4fe3b4537bdfeb4be21b379c5 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 31 12:33:44.822379 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27532442-4616-4d1e-80bd-58872ec000e4 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:44.822671 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27532442-4616-4d1e-80bd-58872ec000e4 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:44.822946 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27532442-4616-4d1e-80bd-58872ec000e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:44.845307 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6617e56-8c4e-444c-82db-3d4568b7d169 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d6d7ea077d49e8a66c6c6762499702', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:44.846028 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6617e56-8c4e-444c-82db-3d4568b7d169 service cinder] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:44.846513 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6617e56-8c4e-444c-82db-3d4568b7d169 service cinder] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:44.846862 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6617e56-8c4e-444c-82db-3d4568b7d169 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:44.851680 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-27532442-4616-4d1e-80bd-58872ec000e4 None None] MFA Rules not processed for user `27fe920f2bee4c4f8133b762e7b9e8dd`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:33:44.853002 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-27532442-4616-4d1e-80bd-58872ec000e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:44.913333 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6617e56-8c4e-444c-82db-3d4568b7d169 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:44.914762 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6617e56-8c4e-444c-82db-3d4568b7d169 service cinder] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:44.924747 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 798/1604] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:33:44 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 31 12:33:44.941514 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6617e56-8c4e-444c-82db-3d4568b7d169 service cinder] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:44.945248 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 807/1605] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:33:44 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 31 12:33:44.958972 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27532442-4616-4d1e-80bd-58872ec000e4 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:44.961442 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-27532442-4616-4d1e-80bd-58872ec000e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:44.966344 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6617e56-8c4e-444c-82db-3d4568b7d169 service cinder] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:45.019294 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:33:45.047398 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27532442-4616-4d1e-80bd-58872ec000e4 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:45.053675 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945a8465-5c96-4a2d-a76f-2d18f495d959 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d6d7ea077d49e8a66c6c6762499702', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:45.054402 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945a8465-5c96-4a2d-a76f-2d18f495d959 service cinder] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:45.054768 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945a8465-5c96-4a2d-a76f-2d18f495d959 service cinder] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:45.055296 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945a8465-5c96-4a2d-a76f-2d18f495d959 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:45.066938 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945a8465-5c96-4a2d-a76f-2d18f495d959 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:45.068479 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945a8465-5c96-4a2d-a76f-2d18f495d959 service cinder] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:45.079387 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caff036b-72c0-4d30-a96e-68197c6f3e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:45.080112 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caff036b-72c0-4d30-a96e-68197c6f3e2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:45.080743 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 808/1606] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:33:44 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 31 12:33:45.080903 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caff036b-72c0-4d30-a96e-68197c6f3e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:45.081173 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caff036b-72c0-4d30-a96e-68197c6f3e2e admin admin] PATH_INFO: `/v3/users/d1a39c9218d744c89085e65cb3e07819` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:45.082347 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caff036b-72c0-4d30-a96e-68197c6f3e2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1a39c9218d744c89085e65cb3e07819)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:45.084308 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caff036b-72c0-4d30-a96e-68197c6f3e2e admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:45.147207 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-caff036b-72c0-4d30-a96e-68197c6f3e2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1a39c9218d744c89085e65cb3e07819'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:45.157509 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-caff036b-72c0-4d30-a96e-68197c6f3e2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1a39c9218d744c89085e65cb3e07819'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:45.158375 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-caff036b-72c0-4d30-a96e-68197c6f3e2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1a39c9218d744c89085e65cb3e07819'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:45.176529 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-caff036b-72c0-4d30-a96e-68197c6f3e2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1a39c9218d744c89085e65cb3e07819'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:45.186512 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-caff036b-72c0-4d30-a96e-68197c6f3e2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1a39c9218d744c89085e65cb3e07819'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:45.200940 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 799/1607] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:33:44 2022] DELETE /identity/v3/users/d1a39c9218d744c89085e65cb3e07819 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:33:45.205505 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22804bf-e6af-4ec8-92b1-e081e092742a None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:45.206512 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22804bf-e6af-4ec8-92b1-e081e092742a None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:45.206791 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22804bf-e6af-4ec8-92b1-e081e092742a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:45.234229 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-f22804bf-e6af-4ec8-92b1-e081e092742a None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:33:45.235675 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-f22804bf-e6af-4ec8-92b1-e081e092742a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:45.273624 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:33:45.310674 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 809/1608] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:33:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:33:45.320988 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caff036b-72c0-4d30-a96e-68197c6f3e2e admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:45.322465 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-caff036b-72c0-4d30-a96e-68197c6f3e2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:45.386563 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caff036b-72c0-4d30-a96e-68197c6f3e2e admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:45.390450 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89521271-66eb-49a2-8a86-49825df20c5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:45.391059 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89521271-66eb-49a2-8a86-49825df20c5f service neutron] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:45.391402 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89521271-66eb-49a2-8a86-49825df20c5f service neutron] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:45.391613 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89521271-66eb-49a2-8a86-49825df20c5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:45.401908 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89521271-66eb-49a2-8a86-49825df20c5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:45.403408 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89521271-66eb-49a2-8a86-49825df20c5f service neutron] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:45.419227 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 800/1609] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:33:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:33:45.745477 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22804bf-e6af-4ec8-92b1-e081e092742a None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:45.747153 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-f22804bf-e6af-4ec8-92b1-e081e092742a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:45.771492 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22804bf-e6af-4ec8-92b1-e081e092742a None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:45.792490 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a413668-bffb-4a61-91ee-4bc4c6aec19e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:45.793356 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a413668-bffb-4a61-91ee-4bc4c6aec19e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:45.793854 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a413668-bffb-4a61-91ee-4bc4c6aec19e admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:45.794277 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a413668-bffb-4a61-91ee-4bc4c6aec19e admin admin] PATH_INFO: `/v3/projects/7167996586f14b54aeb833c5615e290f` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:45.796145 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a413668-bffb-4a61-91ee-4bc4c6aec19e admin admin] RBAC: Authorizing `identity:delete_project(project_id=7167996586f14b54aeb833c5615e290f)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:45.798270 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a413668-bffb-4a61-91ee-4bc4c6aec19e admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:45.814023 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89521271-66eb-49a2-8a86-49825df20c5f service neutron] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:45.816180 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-89521271-66eb-49a2-8a86-49825df20c5f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:45.838402 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89521271-66eb-49a2-8a86-49825df20c5f service neutron] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:45.844380 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa25a68a-ca3a-415f-a708-fb18d772f31f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:45.845123 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa25a68a-ca3a-415f-a708-fb18d772f31f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:45.845475 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa25a68a-ca3a-415f-a708-fb18d772f31f admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:45.845952 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa25a68a-ca3a-415f-a708-fb18d772f31f admin admin] PATH_INFO: `/v3/users/1e871325d6ca43398cf883cfe53914d1` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:45.847214 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa25a68a-ca3a-415f-a708-fb18d772f31f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e871325d6ca43398cf883cfe53914d1)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:45.849645 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa25a68a-ca3a-415f-a708-fb18d772f31f admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:45.868467 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-0a413668-bffb-4a61-91ee-4bc4c6aec19e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7167996586f14b54aeb833c5615e290f', 'request_id': 'req-0a413668-bffb-4a61-91ee-4bc4c6aec19e'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:45.876952 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-0a413668-bffb-4a61-91ee-4bc4c6aec19e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7167996586f14b54aeb833c5615e290f', 'request_id': 'req-0a413668-bffb-4a61-91ee-4bc4c6aec19e'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:45.886697 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-0a413668-bffb-4a61-91ee-4bc4c6aec19e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7167996586f14b54aeb833c5615e290f', 'request_id': 'req-0a413668-bffb-4a61-91ee-4bc4c6aec19e'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:45.914614 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-0a413668-bffb-4a61-91ee-4bc4c6aec19e admin admin] The token cache is being invalidate because project 7167996586f14b54aeb833c5615e290f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:33:45.915132 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-0a413668-bffb-4a61-91ee-4bc4c6aec19e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:45.915931 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-0a413668-bffb-4a61-91ee-4bc4c6aec19e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:45.917874 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 810/1610] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:33:45 2022] DELETE /identity/v3/projects/7167996586f14b54aeb833c5615e290f => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:33:45.925882 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab34249-b8c6-4920-9992-d8445176022e None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:45.926244 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab34249-b8c6-4920-9992-d8445176022e None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:45.926580 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab34249-b8c6-4920-9992-d8445176022e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:45.935432 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-fa25a68a-ca3a-415f-a708-fb18d772f31f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e871325d6ca43398cf883cfe53914d1'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:45.938474 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:33:45.945358 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-fa25a68a-ca3a-415f-a708-fb18d772f31f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e871325d6ca43398cf883cfe53914d1'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:45.946432 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-fa25a68a-ca3a-415f-a708-fb18d772f31f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e871325d6ca43398cf883cfe53914d1'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:45.951795 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-0ab34249-b8c6-4920-9992-d8445176022e None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:33:45.953075 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-0ab34249-b8c6-4920-9992-d8445176022e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:45.960423 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-fa25a68a-ca3a-415f-a708-fb18d772f31f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e871325d6ca43398cf883cfe53914d1'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:45.968364 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-fa25a68a-ca3a-415f-a708-fb18d772f31f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e871325d6ca43398cf883cfe53914d1'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:45.975059 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 801/1611] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:33:45 2022] DELETE /identity/v3/users/1e871325d6ca43398cf883cfe53914d1 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:33:45.979314 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0de692e-3658-4b5f-9f24-544834f0af6e None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:45.979602 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0de692e-3658-4b5f-9f24-544834f0af6e None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:45.979873 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0de692e-3658-4b5f-9f24-544834f0af6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:45.999266 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-c0de692e-3658-4b5f-9f24-544834f0af6e None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:33:46.000513 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-c0de692e-3658-4b5f-9f24-544834f0af6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:46.015324 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 811/1612] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:33:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 31 12:33:46.023540 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adea968-f7fa-41ef-bc74-53c68d699b7b None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:46.024006 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adea968-f7fa-41ef-bc74-53c68d699b7b None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:46.024280 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adea968-f7fa-41ef-bc74-53c68d699b7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:46.046485 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-2adea968-f7fa-41ef-bc74-53c68d699b7b None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:33:46.047727 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-2adea968-f7fa-41ef-bc74-53c68d699b7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:46.063815 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 802/1613] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:33:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 31 12:33:46.073560 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0de692e-3658-4b5f-9f24-544834f0af6e None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:46.075042 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-c0de692e-3658-4b5f-9f24-544834f0af6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:46.111057 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:33:46.138707 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0de692e-3658-4b5f-9f24-544834f0af6e None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:46.138892 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 812/1614] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:33:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 31 12:33:46.142422 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff225794-d6ee-4b57-9410-9fd696d820ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:46.143092 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff225794-d6ee-4b57-9410-9fd696d820ff service neutron] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:46.143316 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adea968-f7fa-41ef-bc74-53c68d699b7b None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:46.143507 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff225794-d6ee-4b57-9410-9fd696d820ff service neutron] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:46.143675 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff225794-d6ee-4b57-9410-9fd696d820ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:46.154344 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff225794-d6ee-4b57-9410-9fd696d820ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:46.155804 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adea968-f7fa-41ef-bc74-53c68d699b7b None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:46.155937 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff225794-d6ee-4b57-9410-9fd696d820ff service neutron] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:46.161228 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a57754-7d23-4b77-98b3-4437311f76e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:46.161911 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a57754-7d23-4b77-98b3-4437311f76e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:46.162391 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a57754-7d23-4b77-98b3-4437311f76e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:46.162768 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a57754-7d23-4b77-98b3-4437311f76e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:46.163535 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a57754-7d23-4b77-98b3-4437311f76e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:33:46.163901 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a57754-7d23-4b77-98b3-4437311f76e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:46.165652 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a57754-7d23-4b77-98b3-4437311f76e7 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:46.168896 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 803/1615] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:33:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 31 12:33:46.180388 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 813/1616] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Oct 31 12:33:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 31 12:33:46.184672 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff225794-d6ee-4b57-9410-9fd696d820ff service neutron] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:46.194871 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff225794-d6ee-4b57-9410-9fd696d820ff service neutron] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:46.200500 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310e7158-2e30-47e0-906e-a46c542b9c44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:46.201116 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310e7158-2e30-47e0-906e-a46c542b9c44 admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:46.201397 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310e7158-2e30-47e0-906e-a46c542b9c44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:46.201667 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310e7158-2e30-47e0-906e-a46c542b9c44 admin admin] PATH_INFO: `/v3/projects` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:46.202292 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310e7158-2e30-47e0-906e-a46c542b9c44 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:46.204228 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310e7158-2e30-47e0-906e-a46c542b9c44 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:46.239208 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 804/1617] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:33:46 2022] POST /identity/v3/projects => generated 383 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:33:46.242010 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a57754-7d23-4b77-98b3-4437311f76e7 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:46.255595 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a57754-7d23-4b77-98b3-4437311f76e7 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:46.319943 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4b8eab-024d-4999-97c9-21904212d56a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:46.320915 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4b8eab-024d-4999-97c9-21904212d56a admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:46.321316 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4b8eab-024d-4999-97c9-21904212d56a admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:46.321821 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4b8eab-024d-4999-97c9-21904212d56a admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:46.322537 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4b8eab-024d-4999-97c9-21904212d56a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:46.324615 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4b8eab-024d-4999-97c9-21904212d56a admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:46.349534 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 814/1618] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:33:46 2022] POST /identity/v3/users => generated 292 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:33:46.353786 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310e7158-2e30-47e0-906e-a46c542b9c44 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:46.366521 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310e7158-2e30-47e0-906e-a46c542b9c44 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:46.371181 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f486b8b3-7f5a-4a60-b356-c49561cfa109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:46.371866 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f486b8b3-7f5a-4a60-b356-c49561cfa109 admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:46.372230 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f486b8b3-7f5a-4a60-b356-c49561cfa109 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:46.372568 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f486b8b3-7f5a-4a60-b356-c49561cfa109 admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:46.373541 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f486b8b3-7f5a-4a60-b356-c49561cfa109 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:46.375233 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f486b8b3-7f5a-4a60-b356-c49561cfa109 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:46.387207 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 805/1619] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:33:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:33:46.391028 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4b8eab-024d-4999-97c9-21904212d56a admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:46.401587 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4b8eab-024d-4999-97c9-21904212d56a admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:46.406755 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fd9b26-0a4d-4e96-9ac1-b19ba62a5269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:46.407311 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fd9b26-0a4d-4e96-9ac1-b19ba62a5269 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:46.407660 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fd9b26-0a4d-4e96-9ac1-b19ba62a5269 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:46.407997 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fd9b26-0a4d-4e96-9ac1-b19ba62a5269 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:46.408910 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fd9b26-0a4d-4e96-9ac1-b19ba62a5269 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:46.410735 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fd9b26-0a4d-4e96-9ac1-b19ba62a5269 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:46.425738 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 815/1620] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:33:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:33:46.427734 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f486b8b3-7f5a-4a60-b356-c49561cfa109 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:46.438061 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f486b8b3-7f5a-4a60-b356-c49561cfa109 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:46.443068 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a004cfa-680f-49c1-bf3f-1a66fab06977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:46.443764 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a004cfa-680f-49c1-bf3f-1a66fab06977 admin admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:46.444046 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a004cfa-680f-49c1-bf3f-1a66fab06977 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:46.444319 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a004cfa-680f-49c1-bf3f-1a66fab06977 admin admin] PATH_INFO: `/v3/projects/63d191c3102a4e7b8ac74cbea9f3a8db/users/31de3cb320d5493dab572757bdafa39d/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:46.460136 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a004cfa-680f-49c1-bf3f-1a66fab06977 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63d191c3102a4e7b8ac74cbea9f3a8db, user_id=31de3cb320d5493dab572757bdafa39d, role_id=ca1a5695f8bd499aa0b8cedaeecd2cc6)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:46.462811 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a004cfa-680f-49c1-bf3f-1a66fab06977 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:46.477739 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 806/1621] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:33:46 2022] PUT /identity/v3/projects/63d191c3102a4e7b8ac74cbea9f3a8db/users/31de3cb320d5493dab572757bdafa39d/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:33:46.489234 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5267ab88-1407-441e-b65f-32f852aafe60 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:46.489343 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5267ab88-1407-441e-b65f-32f852aafe60 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:46.489612 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5267ab88-1407-441e-b65f-32f852aafe60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:46.524463 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-5267ab88-1407-441e-b65f-32f852aafe60 None None] MFA Rules not processed for user `31de3cb320d5493dab572757bdafa39d`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:33:46.525692 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-5267ab88-1407-441e-b65f-32f852aafe60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:46.538477 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a004cfa-680f-49c1-bf3f-1a66fab06977 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:46.540412 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-8a004cfa-680f-49c1-bf3f-1a66fab06977 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:46.559028 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:33:46.608192 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 816/1622] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:33:46 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 31 12:33:46.612696 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5267ab88-1407-441e-b65f-32f852aafe60 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:46.642285 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a004cfa-680f-49c1-bf3f-1a66fab06977 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:46.648352 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c85937c-95ff-4748-a28c-a3875e7373ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:46.649004 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c85937c-95ff-4748-a28c-a3875e7373ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:46.649268 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c85937c-95ff-4748-a28c-a3875e7373ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:46.649535 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c85937c-95ff-4748-a28c-a3875e7373ed admin admin] PATH_INFO: `/v3/projects/76a044019c324ceb898e237b1aa92330` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:46.650665 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c85937c-95ff-4748-a28c-a3875e7373ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=76a044019c324ceb898e237b1aa92330)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:46.652606 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c85937c-95ff-4748-a28c-a3875e7373ed admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:46.683781 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:33:46.700424 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5267ab88-1407-441e-b65f-32f852aafe60 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:46.705522 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40ec575-4357-41b9-a707-bf0af8bc6004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:46.706314 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40ec575-4357-41b9-a707-bf0af8bc6004 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:46.706703 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40ec575-4357-41b9-a707-bf0af8bc6004 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:46.707062 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40ec575-4357-41b9-a707-bf0af8bc6004 admin admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:46.707861 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40ec575-4357-41b9-a707-bf0af8bc6004 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:46.709918 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40ec575-4357-41b9-a707-bf0af8bc6004 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:46.744102 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 817/1623] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:33:46 2022] POST /identity/v3/projects => generated 385 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 31 12:33:46.748666 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40ec575-4357-41b9-a707-bf0af8bc6004 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:46.829485 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-3c85937c-95ff-4748-a28c-a3875e7373ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76a044019c324ceb898e237b1aa92330', 'request_id': 'req-3c85937c-95ff-4748-a28c-a3875e7373ed'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:46.835773 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-3c85937c-95ff-4748-a28c-a3875e7373ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76a044019c324ceb898e237b1aa92330', 'request_id': 'req-3c85937c-95ff-4748-a28c-a3875e7373ed'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:46.837333 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40ec575-4357-41b9-a707-bf0af8bc6004 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:46.844461 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-3c85937c-95ff-4748-a28c-a3875e7373ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76a044019c324ceb898e237b1aa92330', 'request_id': 'req-3c85937c-95ff-4748-a28c-a3875e7373ed'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:46.860639 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-3c85937c-95ff-4748-a28c-a3875e7373ed admin admin] The token cache is being invalidate because project 76a044019c324ceb898e237b1aa92330 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:33:46.861152 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-3c85937c-95ff-4748-a28c-a3875e7373ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:46.861874 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-3c85937c-95ff-4748-a28c-a3875e7373ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:46.863947 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 807/1624] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:33:46 2022] DELETE /identity/v3/projects/76a044019c324ceb898e237b1aa92330 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 31 12:33:46.870216 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c85937c-95ff-4748-a28c-a3875e7373ed admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:46.872293 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-3c85937c-95ff-4748-a28c-a3875e7373ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:46.904776 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4ae4a0-f27e-49ec-bf9e-5f5f0316ff52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:46.905558 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4ae4a0-f27e-49ec-bf9e-5f5f0316ff52 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:46.905957 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4ae4a0-f27e-49ec-bf9e-5f5f0316ff52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:46.906300 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4ae4a0-f27e-49ec-bf9e-5f5f0316ff52 admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:46.907022 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4ae4a0-f27e-49ec-bf9e-5f5f0316ff52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:46.909783 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4ae4a0-f27e-49ec-bf9e-5f5f0316ff52 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:46.940215 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 818/1625] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:33:46 2022] POST /identity/v3/users => generated 293 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:33:46.941169 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4ae4a0-f27e-49ec-bf9e-5f5f0316ff52 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:46.942423 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-4b4ae4a0-f27e-49ec-bf9e-5f5f0316ff52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:46.961215 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4ae4a0-f27e-49ec-bf9e-5f5f0316ff52 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:46.969520 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4159df8c-fb0e-476a-ad1c-c6c3019bc4d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:46.969520 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4159df8c-fb0e-476a-ad1c-c6c3019bc4d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:46.969815 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c85937c-95ff-4748-a28c-a3875e7373ed admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:46.969924 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4159df8c-fb0e-476a-ad1c-c6c3019bc4d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:46.969924 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4159df8c-fb0e-476a-ad1c-c6c3019bc4d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:46.971523 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4159df8c-fb0e-476a-ad1c-c6c3019bc4d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:46.973126 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4159df8c-fb0e-476a-ad1c-c6c3019bc4d1 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:46.975819 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a40c709-5672-422f-8c79-3be8b1a04db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:46.976647 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a40c709-5672-422f-8c79-3be8b1a04db2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:46.977014 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a40c709-5672-422f-8c79-3be8b1a04db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:46.977378 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a40c709-5672-422f-8c79-3be8b1a04db2 admin admin] PATH_INFO: `/v3/users/27fe920f2bee4c4f8133b762e7b9e8dd` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:46.978854 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a40c709-5672-422f-8c79-3be8b1a04db2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27fe920f2bee4c4f8133b762e7b9e8dd)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:46.985182 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a40c709-5672-422f-8c79-3be8b1a04db2 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:47.001903 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 819/1626] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:33:46 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 31 12:33:47.006781 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4159df8c-fb0e-476a-ad1c-c6c3019bc4d1 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:47.018495 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4159df8c-fb0e-476a-ad1c-c6c3019bc4d1 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:47.035366 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40edb6c3-2ca3-46f3-a921-773631278fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:47.036089 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40edb6c3-2ca3-46f3-a921-773631278fab admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:47.036458 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40edb6c3-2ca3-46f3-a921-773631278fab admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:47.036638 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40edb6c3-2ca3-46f3-a921-773631278fab admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:47.039624 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40edb6c3-2ca3-46f3-a921-773631278fab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:47.041453 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40edb6c3-2ca3-46f3-a921-773631278fab admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:47.054768 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 820/1627] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:33:47 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 31 12:33:47.058744 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40edb6c3-2ca3-46f3-a921-773631278fab admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:47.078833 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40edb6c3-2ca3-46f3-a921-773631278fab admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:47.092029 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:33:47.096650 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-3a40c709-5672-422f-8c79-3be8b1a04db2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27fe920f2bee4c4f8133b762e7b9e8dd'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:47.120379 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-3a40c709-5672-422f-8c79-3be8b1a04db2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27fe920f2bee4c4f8133b762e7b9e8dd'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:47.121187 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-3a40c709-5672-422f-8c79-3be8b1a04db2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27fe920f2bee4c4f8133b762e7b9e8dd'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:47.122028 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538b4c15-7c45-4986-a7ee-042743732c07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:47.126644 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538b4c15-7c45-4986-a7ee-042743732c07 admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:47.128005 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538b4c15-7c45-4986-a7ee-042743732c07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:47.128433 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538b4c15-7c45-4986-a7ee-042743732c07 admin admin] PATH_INFO: `/v3/projects/fcd5c7c179ca41db9e897ec17c515e72/users/bb9e3c9c9f234cdb939bb21d12e2237c/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:47.159850 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-3a40c709-5672-422f-8c79-3be8b1a04db2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27fe920f2bee4c4f8133b762e7b9e8dd'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:47.182213 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-3a40c709-5672-422f-8c79-3be8b1a04db2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27fe920f2bee4c4f8133b762e7b9e8dd'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:47.184515 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538b4c15-7c45-4986-a7ee-042743732c07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcd5c7c179ca41db9e897ec17c515e72, user_id=bb9e3c9c9f234cdb939bb21d12e2237c, role_id=ca1a5695f8bd499aa0b8cedaeecd2cc6)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:47.191937 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538b4c15-7c45-4986-a7ee-042743732c07 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:47.197863 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 808/1628] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:33:46 2022] DELETE /identity/v3/users/27fe920f2bee4c4f8133b762e7b9e8dd => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:33:47.210367 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 821/1629] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:33:47 2022] PUT /identity/v3/projects/fcd5c7c179ca41db9e897ec17c515e72/users/bb9e3c9c9f234cdb939bb21d12e2237c/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:33:47.222250 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba884c86-c26c-48ae-b7a3-6a64e070db75 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:47.223512 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba884c86-c26c-48ae-b7a3-6a64e070db75 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:47.224322 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba884c86-c26c-48ae-b7a3-6a64e070db75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:47.287812 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-ba884c86-c26c-48ae-b7a3-6a64e070db75 None None] MFA Rules not processed for user `bb9e3c9c9f234cdb939bb21d12e2237c`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:33:47.291206 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-ba884c86-c26c-48ae-b7a3-6a64e070db75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:47.394766 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 809/1630] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:33:47 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:33:47.399992 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538b4c15-7c45-4986-a7ee-042743732c07 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:47.402161 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-538b4c15-7c45-4986-a7ee-042743732c07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:47.454089 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:33:47.535593 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538b4c15-7c45-4986-a7ee-042743732c07 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:47.542905 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8978605a-e8ab-4e6d-b857-816e3c9b8c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:47.543810 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8978605a-e8ab-4e6d-b857-816e3c9b8c6f admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:47.544194 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8978605a-e8ab-4e6d-b857-816e3c9b8c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:47.544650 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8978605a-e8ab-4e6d-b857-816e3c9b8c6f admin admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:47.546197 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8978605a-e8ab-4e6d-b857-816e3c9b8c6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:47.550218 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8978605a-e8ab-4e6d-b857-816e3c9b8c6f admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:47.640575 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 822/1631] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:33:47 2022] POST /identity/v3/projects => generated 385 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 31 12:33:47.645221 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba884c86-c26c-48ae-b7a3-6a64e070db75 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:47.664911 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba884c86-c26c-48ae-b7a3-6a64e070db75 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:47.752722 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939a0eb8-fd71-49e6-a7e8-2fab488ac273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:47.753449 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939a0eb8-fd71-49e6-a7e8-2fab488ac273 admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:47.753729 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939a0eb8-fd71-49e6-a7e8-2fab488ac273 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:47.754000 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939a0eb8-fd71-49e6-a7e8-2fab488ac273 admin admin] PATH_INFO: `/v3/users` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:47.754778 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939a0eb8-fd71-49e6-a7e8-2fab488ac273 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:47.756864 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939a0eb8-fd71-49e6-a7e8-2fab488ac273 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:47.786680 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8978605a-e8ab-4e6d-b857-816e3c9b8c6f admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:47.787257 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 810/1632] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:33:47 2022] POST /identity/v3/users => generated 299 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:33:47.799820 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8978605a-e8ab-4e6d-b857-816e3c9b8c6f admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:47.805639 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e1bd86-a079-44e6-addd-75ae1c3865f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:47.806403 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e1bd86-a079-44e6-addd-75ae1c3865f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:47.806741 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e1bd86-a079-44e6-addd-75ae1c3865f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:47.807073 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e1bd86-a079-44e6-addd-75ae1c3865f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:47.807817 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e1bd86-a079-44e6-addd-75ae1c3865f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:47.809714 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e1bd86-a079-44e6-addd-75ae1c3865f5 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:47.823042 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 823/1633] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:33:47 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 31 12:33:47.827595 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939a0eb8-fd71-49e6-a7e8-2fab488ac273 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:47.839916 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939a0eb8-fd71-49e6-a7e8-2fab488ac273 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:47.845351 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6eede7-5d8d-4a4b-b846-a99d6632c8c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:47.846944 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6eede7-5d8d-4a4b-b846-a99d6632c8c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:47.846944 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6eede7-5d8d-4a4b-b846-a99d6632c8c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:47.846944 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6eede7-5d8d-4a4b-b846-a99d6632c8c1 admin admin] PATH_INFO: `/v3/projects/8cae26d8ae7f412292629591048c675f/users/170b5ff87168436693104d2478e9c0ac/roles/176c07b4fe3b4537bdfeb4be21b379c5` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:47.865194 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6eede7-5d8d-4a4b-b846-a99d6632c8c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cae26d8ae7f412292629591048c675f, user_id=170b5ff87168436693104d2478e9c0ac, role_id=176c07b4fe3b4537bdfeb4be21b379c5)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:47.867936 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6eede7-5d8d-4a4b-b846-a99d6632c8c1 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:47.888972 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e1bd86-a079-44e6-addd-75ae1c3865f5 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:47.889863 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 811/1634] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:33:47 2022] PUT /identity/v3/projects/8cae26d8ae7f412292629591048c675f/users/170b5ff87168436693104d2478e9c0ac/roles/176c07b4fe3b4537bdfeb4be21b379c5 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:33:47.903227 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e1bd86-a079-44e6-addd-75ae1c3865f5 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:47.976241 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2f8852-54bc-4906-a3fb-d6b3f6ae594e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:47.976857 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2f8852-54bc-4906-a3fb-d6b3f6ae594e admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:47.977136 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2f8852-54bc-4906-a3fb-d6b3f6ae594e admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:47.977406 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2f8852-54bc-4906-a3fb-d6b3f6ae594e admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:47.978048 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2f8852-54bc-4906-a3fb-d6b3f6ae594e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:47.979865 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2f8852-54bc-4906-a3fb-d6b3f6ae594e admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:47.994291 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 824/1635] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:33:47 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:33:47.995533 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6eede7-5d8d-4a4b-b846-a99d6632c8c1 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:48.012129 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6eede7-5d8d-4a4b-b846-a99d6632c8c1 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:48.019419 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac594cd-5277-43db-8abc-6cf9b8743822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:48.020935 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac594cd-5277-43db-8abc-6cf9b8743822 admin admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:48.027283 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac594cd-5277-43db-8abc-6cf9b8743822 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:48.027283 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac594cd-5277-43db-8abc-6cf9b8743822 admin admin] PATH_INFO: `/v3/projects/8cae26d8ae7f412292629591048c675f/users/170b5ff87168436693104d2478e9c0ac/roles/176c07b4fe3b4537bdfeb4be21b379c5` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:48.037173 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac594cd-5277-43db-8abc-6cf9b8743822 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cae26d8ae7f412292629591048c675f, user_id=170b5ff87168436693104d2478e9c0ac, role_id=176c07b4fe3b4537bdfeb4be21b379c5)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:48.042021 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac594cd-5277-43db-8abc-6cf9b8743822 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:48.053542 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 812/1636] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:33:47 2022] PUT /identity/v3/projects/8cae26d8ae7f412292629591048c675f/users/170b5ff87168436693104d2478e9c0ac/roles/176c07b4fe3b4537bdfeb4be21b379c5 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:33:48.064297 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802ddc16-b8a9-44dd-83a8-955ea1c29088 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:48.064835 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802ddc16-b8a9-44dd-83a8-955ea1c29088 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:48.065278 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802ddc16-b8a9-44dd-83a8-955ea1c29088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:48.106244 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-802ddc16-b8a9-44dd-83a8-955ea1c29088 None None] MFA Rules not processed for user `170b5ff87168436693104d2478e9c0ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:33:48.107891 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-802ddc16-b8a9-44dd-83a8-955ea1c29088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:48.188839 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 825/1637] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:33:48 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:33:48.204266 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac594cd-5277-43db-8abc-6cf9b8743822 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:48.209762 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-4ac594cd-5277-43db-8abc-6cf9b8743822 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:48.217529 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=91,keystone:INSERT=6,keystone:DELETE=3 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:33:48.295913 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac594cd-5277-43db-8abc-6cf9b8743822 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:48.300400 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d61dd2-42fc-4a3d-80d1-25757dd29b72 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d6d7ea077d49e8a66c6c6762499702', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:48.301022 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d61dd2-42fc-4a3d-80d1-25757dd29b72 service cinder] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:48.301303 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d61dd2-42fc-4a3d-80d1-25757dd29b72 service cinder] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:48.301576 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d61dd2-42fc-4a3d-80d1-25757dd29b72 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:48.303718 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-06d61dd2-42fc-4a3d-80d1-25757dd29b72 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:48.352461 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802ddc16-b8a9-44dd-83a8-955ea1c29088 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:48.354094 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-802ddc16-b8a9-44dd-83a8-955ea1c29088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:48.386590 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:33:48.399972 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d61dd2-42fc-4a3d-80d1-25757dd29b72 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:48.401624 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d61dd2-42fc-4a3d-80d1-25757dd29b72 service cinder] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:48.415362 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 813/1638] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:33:48 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:33:48.431307 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802ddc16-b8a9-44dd-83a8-955ea1c29088 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:48.437175 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8ed337-59d8-4664-9963-4e15ba8a7c1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:48.438256 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8ed337-59d8-4664-9963-4e15ba8a7c1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:48.438518 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8ed337-59d8-4664-9963-4e15ba8a7c1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:48.438660 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8ed337-59d8-4664-9963-4e15ba8a7c1b admin admin] PATH_INFO: `/v3/projects/989774f7ec3d47edbb69cd843407272f` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:48.440354 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8ed337-59d8-4664-9963-4e15ba8a7c1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=989774f7ec3d47edbb69cd843407272f)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:48.442530 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8ed337-59d8-4664-9963-4e15ba8a7c1b admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:48.505244 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-5b8ed337-59d8-4664-9963-4e15ba8a7c1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '989774f7ec3d47edbb69cd843407272f', 'request_id': 'req-5b8ed337-59d8-4664-9963-4e15ba8a7c1b'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:48.512652 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-5b8ed337-59d8-4664-9963-4e15ba8a7c1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '989774f7ec3d47edbb69cd843407272f', 'request_id': 'req-5b8ed337-59d8-4664-9963-4e15ba8a7c1b'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:48.526540 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-5b8ed337-59d8-4664-9963-4e15ba8a7c1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '989774f7ec3d47edbb69cd843407272f', 'request_id': 'req-5b8ed337-59d8-4664-9963-4e15ba8a7c1b'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:48.564903 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-5b8ed337-59d8-4664-9963-4e15ba8a7c1b admin admin] The token cache is being invalidate because project 989774f7ec3d47edbb69cd843407272f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:33:48.565964 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-5b8ed337-59d8-4664-9963-4e15ba8a7c1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:48.566859 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-5b8ed337-59d8-4664-9963-4e15ba8a7c1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:48.569475 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 826/1639] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:33:48 2022] DELETE /identity/v3/projects/989774f7ec3d47edbb69cd843407272f => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:33:48.574981 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a099f1-636e-4842-9e95-d3ffde13c54c None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:48.575291 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a099f1-636e-4842-9e95-d3ffde13c54c None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:48.575567 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a099f1-636e-4842-9e95-d3ffde13c54c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:48.597522 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-93a099f1-636e-4842-9e95-d3ffde13c54c None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:33:48.599012 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-93a099f1-636e-4842-9e95-d3ffde13c54c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:48.711227 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 814/1640] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:33:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:33:48.716910 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ec77d0-50c4-4638-9adb-8c5ad891d30d None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:48.717839 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ec77d0-50c4-4638-9adb-8c5ad891d30d None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:48.718431 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ec77d0-50c4-4638-9adb-8c5ad891d30d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:48.741703 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-37ec77d0-50c4-4638-9adb-8c5ad891d30d None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:33:48.743128 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-37ec77d0-50c4-4638-9adb-8c5ad891d30d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:48.764777 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 827/1641] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:33:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:33:48.771148 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a099f1-636e-4842-9e95-d3ffde13c54c None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:48.792396 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a099f1-636e-4842-9e95-d3ffde13c54c None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:48.797948 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622d7217-93c1-4715-aa0f-92a4f10777e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:48.798572 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622d7217-93c1-4715-aa0f-92a4f10777e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:48.799519 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622d7217-93c1-4715-aa0f-92a4f10777e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:48.799895 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622d7217-93c1-4715-aa0f-92a4f10777e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:48.800597 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622d7217-93c1-4715-aa0f-92a4f10777e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:33:48.801776 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622d7217-93c1-4715-aa0f-92a4f10777e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:48.803603 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622d7217-93c1-4715-aa0f-92a4f10777e7 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:48.822322 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 815/1642] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Oct 31 12:33:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 31 12:33:48.823720 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ec77d0-50c4-4638-9adb-8c5ad891d30d None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:48.845603 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ec77d0-50c4-4638-9adb-8c5ad891d30d None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:48.845603 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c948e5e-9b6c-40cd-9e85-8ca4bcadb16f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:48.856246 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c948e5e-9b6c-40cd-9e85-8ca4bcadb16f admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:48.856536 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c948e5e-9b6c-40cd-9e85-8ca4bcadb16f admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:48.856813 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c948e5e-9b6c-40cd-9e85-8ca4bcadb16f admin admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:48.857495 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c948e5e-9b6c-40cd-9e85-8ca4bcadb16f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:48.862293 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c948e5e-9b6c-40cd-9e85-8ca4bcadb16f admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:48.929845 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 828/1643] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:33:48 2022] POST /identity/v3/projects => generated 381 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:33:48.931044 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622d7217-93c1-4715-aa0f-92a4f10777e7 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:48.944755 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622d7217-93c1-4715-aa0f-92a4f10777e7 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:49.032672 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:33:49.041729 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53ae43b-3833-4054-9f5f-907c73735080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:49.042556 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53ae43b-3833-4054-9f5f-907c73735080 admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:49.042900 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53ae43b-3833-4054-9f5f-907c73735080 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:49.043265 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53ae43b-3833-4054-9f5f-907c73735080 admin admin] PATH_INFO: `/v3/users` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:49.043994 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53ae43b-3833-4054-9f5f-907c73735080 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:49.046242 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53ae43b-3833-4054-9f5f-907c73735080 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:49.071448 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 816/1644] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:33:48 2022] POST /identity/v3/users => generated 291 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:33:49.075658 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c948e5e-9b6c-40cd-9e85-8ca4bcadb16f admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:49.089564 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c948e5e-9b6c-40cd-9e85-8ca4bcadb16f admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:49.094499 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f324b28-9f89-4639-930f-0efae473da8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:49.095217 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f324b28-9f89-4639-930f-0efae473da8e admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:49.095502 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f324b28-9f89-4639-930f-0efae473da8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:49.095811 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f324b28-9f89-4639-930f-0efae473da8e admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:49.096686 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f324b28-9f89-4639-930f-0efae473da8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:49.098396 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f324b28-9f89-4639-930f-0efae473da8e admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:49.113922 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 829/1645] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:33:49 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:33:49.115213 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53ae43b-3833-4054-9f5f-907c73735080 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:49.125824 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53ae43b-3833-4054-9f5f-907c73735080 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:49.134378 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30dcdb3-2fec-43da-9611-54aaaaf23d7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:49.135362 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30dcdb3-2fec-43da-9611-54aaaaf23d7c admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:49.135846 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30dcdb3-2fec-43da-9611-54aaaaf23d7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:49.136215 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30dcdb3-2fec-43da-9611-54aaaaf23d7c admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:49.136999 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30dcdb3-2fec-43da-9611-54aaaaf23d7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:49.138725 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30dcdb3-2fec-43da-9611-54aaaaf23d7c admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:49.152858 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 817/1646] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:33:49 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:33:49.156828 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f324b28-9f89-4639-930f-0efae473da8e admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:49.171210 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f324b28-9f89-4639-930f-0efae473da8e admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:49.177082 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3427f879-bddc-435b-b929-2cdafcf64aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:49.178050 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3427f879-bddc-435b-b929-2cdafcf64aa7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:49.178647 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3427f879-bddc-435b-b929-2cdafcf64aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:49.179218 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3427f879-bddc-435b-b929-2cdafcf64aa7 admin admin] PATH_INFO: `/v3/projects/313de9b042a14edd8afc4186bc5f0e7f/users/f707c8dfdbdc478bbf993df3c61bcff2/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:49.198775 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3427f879-bddc-435b-b929-2cdafcf64aa7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=313de9b042a14edd8afc4186bc5f0e7f, user_id=f707c8dfdbdc478bbf993df3c61bcff2, role_id=ca1a5695f8bd499aa0b8cedaeecd2cc6)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:49.201568 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3427f879-bddc-435b-b929-2cdafcf64aa7 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:49.215564 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 830/1647] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:33:49 2022] PUT /identity/v3/projects/313de9b042a14edd8afc4186bc5f0e7f/users/f707c8dfdbdc478bbf993df3c61bcff2/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:33:49.226575 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae3c3d5-6cec-4a8c-b71a-41c27ca853db None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:49.226853 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae3c3d5-6cec-4a8c-b71a-41c27ca853db None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:49.227120 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae3c3d5-6cec-4a8c-b71a-41c27ca853db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:49.256338 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-bae3c3d5-6cec-4a8c-b71a-41c27ca853db None None] MFA Rules not processed for user `f707c8dfdbdc478bbf993df3c61bcff2`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:33:49.257643 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-bae3c3d5-6cec-4a8c-b71a-41c27ca853db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:49.347584 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 818/1648] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:33:49 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:33:49.351760 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3427f879-bddc-435b-b929-2cdafcf64aa7 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:49.411620 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:33:49.426965 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3427f879-bddc-435b-b929-2cdafcf64aa7 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:49.431956 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca59195-47e7-44fb-8198-352589a8b123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:49.432576 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca59195-47e7-44fb-8198-352589a8b123 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:49.433045 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca59195-47e7-44fb-8198-352589a8b123 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:49.433391 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca59195-47e7-44fb-8198-352589a8b123 admin admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:49.434105 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca59195-47e7-44fb-8198-352589a8b123 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:49.436097 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca59195-47e7-44fb-8198-352589a8b123 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:49.465993 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 831/1649] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:33:49 2022] POST /identity/v3/projects => generated 379 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:33:49.470715 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae3c3d5-6cec-4a8c-b71a-41c27ca853db None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:49.483617 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae3c3d5-6cec-4a8c-b71a-41c27ca853db None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:49.555754 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8714e29-bdfb-44d3-83a3-a4c93269c6fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:49.556472 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8714e29-bdfb-44d3-83a3-a4c93269c6fe admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:49.556847 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8714e29-bdfb-44d3-83a3-a4c93269c6fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:49.557115 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8714e29-bdfb-44d3-83a3-a4c93269c6fe admin admin] PATH_INFO: `/v3/users` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:49.557749 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8714e29-bdfb-44d3-83a3-a4c93269c6fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:49.559672 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8714e29-bdfb-44d3-83a3-a4c93269c6fe admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:49.588654 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 819/1650] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:33:49 2022] POST /identity/v3/users => generated 296 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:33:49.592780 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca59195-47e7-44fb-8198-352589a8b123 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:49.606444 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca59195-47e7-44fb-8198-352589a8b123 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:49.611270 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec55b39e-1383-45d5-ab0f-67109177d9df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:49.612476 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec55b39e-1383-45d5-ab0f-67109177d9df admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:49.612764 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec55b39e-1383-45d5-ab0f-67109177d9df admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:49.613030 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec55b39e-1383-45d5-ab0f-67109177d9df admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:49.613674 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec55b39e-1383-45d5-ab0f-67109177d9df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:49.615340 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec55b39e-1383-45d5-ab0f-67109177d9df admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:49.631384 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 832/1651] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:33:49 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:33:49.632430 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8714e29-bdfb-44d3-83a3-a4c93269c6fe admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:49.645823 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8714e29-bdfb-44d3-83a3-a4c93269c6fe admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:49.652319 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4225d0c1-0992-4ab3-8e2b-a086acfe92c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:49.653038 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4225d0c1-0992-4ab3-8e2b-a086acfe92c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:49.653323 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4225d0c1-0992-4ab3-8e2b-a086acfe92c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:49.654341 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4225d0c1-0992-4ab3-8e2b-a086acfe92c3 admin admin] PATH_INFO: `/v3/projects/5be92a3a49a04d678d08b145daed967f/users/8b7ff9e83ca74c6fa73250bfbcf69f09/roles/176c07b4fe3b4537bdfeb4be21b379c5` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:49.670248 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4225d0c1-0992-4ab3-8e2b-a086acfe92c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5be92a3a49a04d678d08b145daed967f, user_id=8b7ff9e83ca74c6fa73250bfbcf69f09, role_id=176c07b4fe3b4537bdfeb4be21b379c5)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:49.672970 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4225d0c1-0992-4ab3-8e2b-a086acfe92c3 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:49.689610 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 820/1652] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:33:49 2022] PUT /identity/v3/projects/5be92a3a49a04d678d08b145daed967f/users/8b7ff9e83ca74c6fa73250bfbcf69f09/roles/176c07b4fe3b4537bdfeb4be21b379c5 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:33:49.693454 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec55b39e-1383-45d5-ab0f-67109177d9df admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:49.704836 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec55b39e-1383-45d5-ab0f-67109177d9df admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:49.773513 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d9aeb5-1688-4aa0-b3f6-843a8616622f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:49.774134 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d9aeb5-1688-4aa0-b3f6-843a8616622f admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:49.774411 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d9aeb5-1688-4aa0-b3f6-843a8616622f admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:49.775538 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d9aeb5-1688-4aa0-b3f6-843a8616622f admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:49.776195 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d9aeb5-1688-4aa0-b3f6-843a8616622f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:49.777809 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d9aeb5-1688-4aa0-b3f6-843a8616622f admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:49.788034 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 833/1653] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:33:49 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:33:49.792122 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4225d0c1-0992-4ab3-8e2b-a086acfe92c3 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:49.802751 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4225d0c1-0992-4ab3-8e2b-a086acfe92c3 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:49.807381 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92223e74-aaca-4170-8397-03dd4cf3aba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:49.808104 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92223e74-aaca-4170-8397-03dd4cf3aba0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:49.808448 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92223e74-aaca-4170-8397-03dd4cf3aba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:49.808896 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92223e74-aaca-4170-8397-03dd4cf3aba0 admin admin] PATH_INFO: `/v3/projects/5be92a3a49a04d678d08b145daed967f/users/8b7ff9e83ca74c6fa73250bfbcf69f09/roles/176c07b4fe3b4537bdfeb4be21b379c5` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:49.810926 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92223e74-aaca-4170-8397-03dd4cf3aba0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5be92a3a49a04d678d08b145daed967f, user_id=8b7ff9e83ca74c6fa73250bfbcf69f09, role_id=176c07b4fe3b4537bdfeb4be21b379c5)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:49.813624 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92223e74-aaca-4170-8397-03dd4cf3aba0 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:49.825192 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 821/1654] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:33:49 2022] PUT /identity/v3/projects/5be92a3a49a04d678d08b145daed967f/users/8b7ff9e83ca74c6fa73250bfbcf69f09/roles/176c07b4fe3b4537bdfeb4be21b379c5 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:33:49.836940 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d102a6-0fc6-482f-aaef-b9e1c5b02a4b None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:49.837470 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d102a6-0fc6-482f-aaef-b9e1c5b02a4b None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:49.837470 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d102a6-0fc6-482f-aaef-b9e1c5b02a4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:49.875922 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-51d102a6-0fc6-482f-aaef-b9e1c5b02a4b None None] MFA Rules not processed for user `8b7ff9e83ca74c6fa73250bfbcf69f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:33:49.877671 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-51d102a6-0fc6-482f-aaef-b9e1c5b02a4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:49.968551 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 834/1655] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:33:49 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:33:49.977994 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92223e74-aaca-4170-8397-03dd4cf3aba0 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:49.980206 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-92223e74-aaca-4170-8397-03dd4cf3aba0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:49.992820 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:33:50.073024 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92223e74-aaca-4170-8397-03dd4cf3aba0 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:50.076894 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9d6e25-e2b3-4dcd-92ef-63ef550919bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d6d7ea077d49e8a66c6c6762499702', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:50.077964 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9d6e25-e2b3-4dcd-92ef-63ef550919bb service cinder] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:50.078242 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9d6e25-e2b3-4dcd-92ef-63ef550919bb service cinder] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:50.078512 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9d6e25-e2b3-4dcd-92ef-63ef550919bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:50.090090 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9d6e25-e2b3-4dcd-92ef-63ef550919bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:50.091576 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9d6e25-e2b3-4dcd-92ef-63ef550919bb service cinder] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:50.110263 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 822/1656] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:33:49 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:33:50.177996 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d102a6-0fc6-482f-aaef-b9e1c5b02a4b None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:50.179505 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-51d102a6-0fc6-482f-aaef-b9e1c5b02a4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:50.194349 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d102a6-0fc6-482f-aaef-b9e1c5b02a4b None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:50.199789 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f380392b-7122-47b5-bbcd-b46aebff2f51 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d6d7ea077d49e8a66c6c6762499702', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:50.200405 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f380392b-7122-47b5-bbcd-b46aebff2f51 service cinder] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:50.200683 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f380392b-7122-47b5-bbcd-b46aebff2f51 service cinder] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:50.200949 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f380392b-7122-47b5-bbcd-b46aebff2f51 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:50.202700 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-f380392b-7122-47b5-bbcd-b46aebff2f51 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:50.254656 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:33:50.280548 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f380392b-7122-47b5-bbcd-b46aebff2f51 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:50.282510 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f380392b-7122-47b5-bbcd-b46aebff2f51 service cinder] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:50.294482 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 835/1657] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:33:50 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:33:50.796790 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9d6e25-e2b3-4dcd-92ef-63ef550919bb service cinder] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:50.886712 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9d6e25-e2b3-4dcd-92ef-63ef550919bb service cinder] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:50.891896 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98915ef6-8e67-42f5-b0fb-b101badde879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:50.893662 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98915ef6-8e67-42f5-b0fb-b101badde879 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:50.894025 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98915ef6-8e67-42f5-b0fb-b101badde879 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:50.894349 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98915ef6-8e67-42f5-b0fb-b101badde879 admin admin] PATH_INFO: `/v3/users/f707c8dfdbdc478bbf993df3c61bcff2` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:50.895924 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98915ef6-8e67-42f5-b0fb-b101badde879 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f707c8dfdbdc478bbf993df3c61bcff2)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:50.897909 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98915ef6-8e67-42f5-b0fb-b101badde879 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:51.000611 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-98915ef6-8e67-42f5-b0fb-b101badde879 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f707c8dfdbdc478bbf993df3c61bcff2'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:51.053438 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-98915ef6-8e67-42f5-b0fb-b101badde879 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f707c8dfdbdc478bbf993df3c61bcff2'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:51.053875 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-98915ef6-8e67-42f5-b0fb-b101badde879 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f707c8dfdbdc478bbf993df3c61bcff2'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:51.071268 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-98915ef6-8e67-42f5-b0fb-b101badde879 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f707c8dfdbdc478bbf993df3c61bcff2'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:51.081323 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-98915ef6-8e67-42f5-b0fb-b101badde879 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f707c8dfdbdc478bbf993df3c61bcff2'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:51.088882 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 823/1658] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:33:50 2022] DELETE /identity/v3/users/f707c8dfdbdc478bbf993df3c61bcff2 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 31 12:33:51.093098 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a29a760-30d6-4706-bf26-1bda9bf30b81 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:51.093576 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a29a760-30d6-4706-bf26-1bda9bf30b81 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:51.093688 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a29a760-30d6-4706-bf26-1bda9bf30b81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:51.114222 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-6a29a760-30d6-4706-bf26-1bda9bf30b81 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:33:51.115864 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-6a29a760-30d6-4706-bf26-1bda9bf30b81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:51.133400 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98915ef6-8e67-42f5-b0fb-b101badde879 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:51.135422 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-98915ef6-8e67-42f5-b0fb-b101badde879 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:51.177238 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:33:51.198502 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 836/1659] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:33:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 31 12:33:51.206564 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a29a760-30d6-4706-bf26-1bda9bf30b81 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:51.208286 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-6a29a760-30d6-4706-bf26-1bda9bf30b81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:51.215258 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98915ef6-8e67-42f5-b0fb-b101badde879 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:51.221971 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e40da9-1a7f-4f33-a7b3-e98cffd3e688 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d6d7ea077d49e8a66c6c6762499702', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:51.222745 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e40da9-1a7f-4f33-a7b3-e98cffd3e688 service cinder] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:51.223144 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e40da9-1a7f-4f33-a7b3-e98cffd3e688 service cinder] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:51.223593 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e40da9-1a7f-4f33-a7b3-e98cffd3e688 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:51.225683 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-42e40da9-1a7f-4f33-a7b3-e98cffd3e688 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:51.293492 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a29a760-30d6-4706-bf26-1bda9bf30b81 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:51.297708 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627509b4-3dcd-4c35-9385-7c4366562374 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:51.298718 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627509b4-3dcd-4c35-9385-7c4366562374 service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:51.299389 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627509b4-3dcd-4c35-9385-7c4366562374 service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:51.299812 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627509b4-3dcd-4c35-9385-7c4366562374 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:51.321339 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627509b4-3dcd-4c35-9385-7c4366562374 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:51.323451 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627509b4-3dcd-4c35-9385-7c4366562374 service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:51.351560 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e40da9-1a7f-4f33-a7b3-e98cffd3e688 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:51.353156 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e40da9-1a7f-4f33-a7b3-e98cffd3e688 service cinder] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:51.353954 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 837/1660] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:33:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 31 12:33:51.371363 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 824/1661] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:33:51 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 31 12:33:51.725662 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627509b4-3dcd-4c35-9385-7c4366562374 service neutron] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:51.728167 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-627509b4-3dcd-4c35-9385-7c4366562374 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:51.752488 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627509b4-3dcd-4c35-9385-7c4366562374 service neutron] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:51.762387 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668ff668-4e25-4ca0-85ac-332bcbb93400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:51.763037 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668ff668-4e25-4ca0-85ac-332bcbb93400 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:51.763707 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668ff668-4e25-4ca0-85ac-332bcbb93400 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:51.764013 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668ff668-4e25-4ca0-85ac-332bcbb93400 admin admin] PATH_INFO: `/v3/users/9463696528534ac5a11d701cd318a3fa` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:51.765541 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668ff668-4e25-4ca0-85ac-332bcbb93400 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9463696528534ac5a11d701cd318a3fa)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:51.768903 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668ff668-4e25-4ca0-85ac-332bcbb93400 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:51.844121 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-668ff668-4e25-4ca0-85ac-332bcbb93400 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9463696528534ac5a11d701cd318a3fa'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:51.867090 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-668ff668-4e25-4ca0-85ac-332bcbb93400 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9463696528534ac5a11d701cd318a3fa'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:51.867942 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-668ff668-4e25-4ca0-85ac-332bcbb93400 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9463696528534ac5a11d701cd318a3fa'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:51.868868 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-668ff668-4e25-4ca0-85ac-332bcbb93400 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9463696528534ac5a11d701cd318a3fa'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:51.898897 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e40da9-1a7f-4f33-a7b3-e98cffd3e688 service cinder] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:51.900522 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-42e40da9-1a7f-4f33-a7b3-e98cffd3e688 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:51.916453 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e40da9-1a7f-4f33-a7b3-e98cffd3e688 service cinder] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:51.916772 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-668ff668-4e25-4ca0-85ac-332bcbb93400 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9463696528534ac5a11d701cd318a3fa'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:51.921611 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c9d894-393e-409b-bbf7-e92297bf8f78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:51.922264 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c9d894-393e-409b-bbf7-e92297bf8f78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:51.922541 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c9d894-393e-409b-bbf7-e92297bf8f78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:51.922879 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c9d894-393e-409b-bbf7-e92297bf8f78 admin admin] PATH_INFO: `/v3/projects/313de9b042a14edd8afc4186bc5f0e7f` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:51.926901 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c9d894-393e-409b-bbf7-e92297bf8f78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=313de9b042a14edd8afc4186bc5f0e7f)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:51.927465 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:33:51.928856 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c9d894-393e-409b-bbf7-e92297bf8f78 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:51.941017 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 838/1662] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:33:51 2022] DELETE /identity/v3/users/9463696528534ac5a11d701cd318a3fa => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:33:51.942474 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a105cf-ceb0-4b1a-866b-855b585c8720 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:51.942995 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a105cf-ceb0-4b1a-866b-855b585c8720 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:51.943306 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a105cf-ceb0-4b1a-866b-855b585c8720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:51.964680 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-77a105cf-ceb0-4b1a-866b-855b585c8720 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:33:51.966024 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-77a105cf-ceb0-4b1a-866b-855b585c8720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:52.017129 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-63c9d894-393e-409b-bbf7-e92297bf8f78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '313de9b042a14edd8afc4186bc5f0e7f', 'request_id': 'req-63c9d894-393e-409b-bbf7-e92297bf8f78'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:52.038597 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-63c9d894-393e-409b-bbf7-e92297bf8f78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '313de9b042a14edd8afc4186bc5f0e7f', 'request_id': 'req-63c9d894-393e-409b-bbf7-e92297bf8f78'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:52.062001 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-63c9d894-393e-409b-bbf7-e92297bf8f78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '313de9b042a14edd8afc4186bc5f0e7f', 'request_id': 'req-63c9d894-393e-409b-bbf7-e92297bf8f78'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:52.085164 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-63c9d894-393e-409b-bbf7-e92297bf8f78 admin admin] The token cache is being invalidate because project 313de9b042a14edd8afc4186bc5f0e7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:33:52.088742 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-63c9d894-393e-409b-bbf7-e92297bf8f78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:52.088882 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 839/1663] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:33:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:33:52.088982 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-63c9d894-393e-409b-bbf7-e92297bf8f78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:52.089256 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 825/1664] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:33:51 2022] DELETE /identity/v3/projects/313de9b042a14edd8afc4186bc5f0e7f => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 31 12:33:52.093293 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a105cf-ceb0-4b1a-866b-855b585c8720 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:52.097552 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-77a105cf-ceb0-4b1a-866b-855b585c8720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:52.100770 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c9d894-393e-409b-bbf7-e92297bf8f78 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:52.102275 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-63c9d894-393e-409b-bbf7-e92297bf8f78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:52.130171 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a105cf-ceb0-4b1a-866b-855b585c8720 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:52.202656 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c9d894-393e-409b-bbf7-e92297bf8f78 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:52.224267 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f2e067-ccbb-4fb3-a06a-62d568775233 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:52.225054 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f2e067-ccbb-4fb3-a06a-62d568775233 service neutron] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:52.225500 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f2e067-ccbb-4fb3-a06a-62d568775233 service neutron] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:52.225800 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f2e067-ccbb-4fb3-a06a-62d568775233 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:52.227563 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-d4f2e067-ccbb-4fb3-a06a-62d568775233 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:52.228127 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ded71a-a34f-4881-ad1d-242e81cab705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:52.229036 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ded71a-a34f-4881-ad1d-242e81cab705 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:52.229036 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ded71a-a34f-4881-ad1d-242e81cab705 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:52.229296 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ded71a-a34f-4881-ad1d-242e81cab705 admin admin] PATH_INFO: `/v3/users/8b7ff9e83ca74c6fa73250bfbcf69f09` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:52.230383 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ded71a-a34f-4881-ad1d-242e81cab705 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b7ff9e83ca74c6fa73250bfbcf69f09)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:52.232362 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ded71a-a34f-4881-ad1d-242e81cab705 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:52.244273 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f2e067-ccbb-4fb3-a06a-62d568775233 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:52.245811 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f2e067-ccbb-4fb3-a06a-62d568775233 service neutron] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:52.259379 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 826/1665] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:33:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 31 12:33:52.286517 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-e5ded71a-a34f-4881-ad1d-242e81cab705 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b7ff9e83ca74c6fa73250bfbcf69f09'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:52.293787 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-e5ded71a-a34f-4881-ad1d-242e81cab705 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b7ff9e83ca74c6fa73250bfbcf69f09'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:52.294434 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-e5ded71a-a34f-4881-ad1d-242e81cab705 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b7ff9e83ca74c6fa73250bfbcf69f09'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:52.295039 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-e5ded71a-a34f-4881-ad1d-242e81cab705 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b7ff9e83ca74c6fa73250bfbcf69f09'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:52.307828 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-e5ded71a-a34f-4881-ad1d-242e81cab705 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b7ff9e83ca74c6fa73250bfbcf69f09'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:52.320852 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 840/1666] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:33:52 2022] DELETE /identity/v3/users/8b7ff9e83ca74c6fa73250bfbcf69f09 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:33:52.648225 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f2e067-ccbb-4fb3-a06a-62d568775233 service neutron] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:52.651024 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-d4f2e067-ccbb-4fb3-a06a-62d568775233 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:52.664625 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:33:52.672442 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f2e067-ccbb-4fb3-a06a-62d568775233 service neutron] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:52.767348 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5c09ef-8dc9-438f-ad76-7866e7580f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:52.768638 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5c09ef-8dc9-438f-ad76-7866e7580f93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:52.769009 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5c09ef-8dc9-438f-ad76-7866e7580f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:52.769295 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5c09ef-8dc9-438f-ad76-7866e7580f93 admin admin] PATH_INFO: `/v3/projects/5be92a3a49a04d678d08b145daed967f` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:52.771260 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5c09ef-8dc9-438f-ad76-7866e7580f93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5be92a3a49a04d678d08b145daed967f)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:52.772798 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5c09ef-8dc9-438f-ad76-7866e7580f93 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:52.839971 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-db5c09ef-8dc9-438f-ad76-7866e7580f93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5be92a3a49a04d678d08b145daed967f', 'request_id': 'req-db5c09ef-8dc9-438f-ad76-7866e7580f93'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:52.849639 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-db5c09ef-8dc9-438f-ad76-7866e7580f93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5be92a3a49a04d678d08b145daed967f', 'request_id': 'req-db5c09ef-8dc9-438f-ad76-7866e7580f93'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:52.859128 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-db5c09ef-8dc9-438f-ad76-7866e7580f93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5be92a3a49a04d678d08b145daed967f', 'request_id': 'req-db5c09ef-8dc9-438f-ad76-7866e7580f93'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:52.876505 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-db5c09ef-8dc9-438f-ad76-7866e7580f93 admin admin] The token cache is being invalidate because project 5be92a3a49a04d678d08b145daed967f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:33:52.878466 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-db5c09ef-8dc9-438f-ad76-7866e7580f93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:52.878618 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-db5c09ef-8dc9-438f-ad76-7866e7580f93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:52.880530 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 827/1667] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:33:52 2022] DELETE /identity/v3/projects/5be92a3a49a04d678d08b145daed967f => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:33:52.886419 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029907e9-4318-4cb5-a027-cc6583a01923 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:52.886708 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029907e9-4318-4cb5-a027-cc6583a01923 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:52.886979 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029907e9-4318-4cb5-a027-cc6583a01923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:52.927780 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-029907e9-4318-4cb5-a027-cc6583a01923 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:33:52.929829 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-029907e9-4318-4cb5-a027-cc6583a01923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:52.956938 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:33:53.071222 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 841/1668] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:33:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 31 12:33:53.078737 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c89e10-72e8-4c4c-97b1-5e465e7312bc None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:53.079285 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c89e10-72e8-4c4c-97b1-5e465e7312bc None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:53.079371 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c89e10-72e8-4c4c-97b1-5e465e7312bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:53.130594 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-35c89e10-72e8-4c4c-97b1-5e465e7312bc None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:33:53.131873 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-35c89e10-72e8-4c4c-97b1-5e465e7312bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:53.153894 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 828/1669] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:33:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:33:53.160096 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029907e9-4318-4cb5-a027-cc6583a01923 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:53.179290 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029907e9-4318-4cb5-a027-cc6583a01923 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:53.198971 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c79d41-459f-4ef2-9c58-601d95b73047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:53.202827 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c79d41-459f-4ef2-9c58-601d95b73047 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:53.203234 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c79d41-459f-4ef2-9c58-601d95b73047 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:53.203661 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c79d41-459f-4ef2-9c58-601d95b73047 admin admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:53.207616 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c79d41-459f-4ef2-9c58-601d95b73047 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:33:53.207941 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c79d41-459f-4ef2-9c58-601d95b73047 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:53.209588 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c79d41-459f-4ef2-9c58-601d95b73047 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:53.269810 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c89e10-72e8-4c4c-97b1-5e465e7312bc None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:53.277174 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 842/1670] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Oct 31 12:33:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:33:53.281289 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c89e10-72e8-4c4c-97b1-5e465e7312bc None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:53.300638 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a78dfc4-de0c-45bb-b679-02945c3f498a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:53.301272 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a78dfc4-de0c-45bb-b679-02945c3f498a admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:53.301551 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a78dfc4-de0c-45bb-b679-02945c3f498a admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:53.301817 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a78dfc4-de0c-45bb-b679-02945c3f498a admin admin] PATH_INFO: `/v3/projects` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:53.303040 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a78dfc4-de0c-45bb-b679-02945c3f498a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:53.304999 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a78dfc4-de0c-45bb-b679-02945c3f498a admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:53.378493 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 829/1671] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:33:53 2022] POST /identity/v3/projects => generated 373 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:33:53.381398 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c79d41-459f-4ef2-9c58-601d95b73047 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:53.397630 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c79d41-459f-4ef2-9c58-601d95b73047 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:53.491607 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f0f8f9-0c59-4aba-be82-1b9c39134773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:53.492416 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f0f8f9-0c59-4aba-be82-1b9c39134773 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:53.492912 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f0f8f9-0c59-4aba-be82-1b9c39134773 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:53.493309 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f0f8f9-0c59-4aba-be82-1b9c39134773 admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:53.494069 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f0f8f9-0c59-4aba-be82-1b9c39134773 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:53.496121 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f0f8f9-0c59-4aba-be82-1b9c39134773 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:53.530152 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 843/1672] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:33:53 2022] POST /identity/v3/users => generated 287 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:33:53.530595 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a78dfc4-de0c-45bb-b679-02945c3f498a admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:53.545004 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a78dfc4-de0c-45bb-b679-02945c3f498a admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:53.551201 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0019474-060c-47d5-8d1e-32681359839b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:53.551869 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0019474-060c-47d5-8d1e-32681359839b admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:53.552143 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0019474-060c-47d5-8d1e-32681359839b admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:53.552407 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0019474-060c-47d5-8d1e-32681359839b admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:53.553050 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0019474-060c-47d5-8d1e-32681359839b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:53.554674 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0019474-060c-47d5-8d1e-32681359839b admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:53.555275 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f0f8f9-0c59-4aba-be82-1b9c39134773 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:53.557239 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-84f0f8f9-0c59-4aba-be82-1b9c39134773 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:53.567886 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 830/1673] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:33:53 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:33:53.572553 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0019474-060c-47d5-8d1e-32681359839b admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:53.584079 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f0f8f9-0c59-4aba-be82-1b9c39134773 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:53.593619 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aecbd91-52bc-4be1-afbf-8a08598532c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:53.594513 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aecbd91-52bc-4be1-afbf-8a08598532c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:53.594928 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aecbd91-52bc-4be1-afbf-8a08598532c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:53.595538 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aecbd91-52bc-4be1-afbf-8a08598532c2 admin admin] PATH_INFO: `/v3/projects/e32241f95a6146d5b216e8208ea02029` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:53.600858 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aecbd91-52bc-4be1-afbf-8a08598532c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e32241f95a6146d5b216e8208ea02029)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:53.601328 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0019474-060c-47d5-8d1e-32681359839b admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:53.605658 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aecbd91-52bc-4be1-afbf-8a08598532c2 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:53.613471 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfd9cbe-5af7-4c29-b14b-0aef3eab4831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:53.616354 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfd9cbe-5af7-4c29-b14b-0aef3eab4831 admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:53.616713 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfd9cbe-5af7-4c29-b14b-0aef3eab4831 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:53.617278 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfd9cbe-5af7-4c29-b14b-0aef3eab4831 admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:53.632214 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfd9cbe-5af7-4c29-b14b-0aef3eab4831 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:53.635537 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfd9cbe-5af7-4c29-b14b-0aef3eab4831 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:53.659445 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 831/1674] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:33:53 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 31 12:33:53.665320 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfd9cbe-5af7-4c29-b14b-0aef3eab4831 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:53.680696 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfd9cbe-5af7-4c29-b14b-0aef3eab4831 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:53.693088 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924eed4a-1898-4286-9437-eab76de057e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:53.694073 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924eed4a-1898-4286-9437-eab76de057e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:53.694682 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924eed4a-1898-4286-9437-eab76de057e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:53.695483 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924eed4a-1898-4286-9437-eab76de057e6 admin admin] PATH_INFO: `/v3/projects/a59db1f0925a439a910182891aa6ae6b/users/255e4b6fc026453e97de997ec6ab03f7/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:53.722492 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-1aecbd91-52bc-4be1-afbf-8a08598532c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e32241f95a6146d5b216e8208ea02029', 'request_id': 'req-1aecbd91-52bc-4be1-afbf-8a08598532c2'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:53.724497 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924eed4a-1898-4286-9437-eab76de057e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a59db1f0925a439a910182891aa6ae6b, user_id=255e4b6fc026453e97de997ec6ab03f7, role_id=ca1a5695f8bd499aa0b8cedaeecd2cc6)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:53.727422 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924eed4a-1898-4286-9437-eab76de057e6 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:53.731808 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-1aecbd91-52bc-4be1-afbf-8a08598532c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e32241f95a6146d5b216e8208ea02029', 'request_id': 'req-1aecbd91-52bc-4be1-afbf-8a08598532c2'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:53.742144 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-1aecbd91-52bc-4be1-afbf-8a08598532c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e32241f95a6146d5b216e8208ea02029', 'request_id': 'req-1aecbd91-52bc-4be1-afbf-8a08598532c2'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:53.745161 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 832/1675] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:33:53 2022] PUT /identity/v3/projects/a59db1f0925a439a910182891aa6ae6b/users/255e4b6fc026453e97de997ec6ab03f7/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:33:53.755387 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-1aecbd91-52bc-4be1-afbf-8a08598532c2 admin admin] The token cache is being invalidate because project e32241f95a6146d5b216e8208ea02029 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:33:53.755682 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-1aecbd91-52bc-4be1-afbf-8a08598532c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:53.757182 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-1aecbd91-52bc-4be1-afbf-8a08598532c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:53.758841 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 844/1676] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:33:53 2022] DELETE /identity/v3/projects/e32241f95a6146d5b216e8208ea02029 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:33:53.763897 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb68b67-dbcb-49ce-940b-727b666008eb None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:53.764515 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb68b67-dbcb-49ce-940b-727b666008eb None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:53.764902 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb68b67-dbcb-49ce-940b-727b666008eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:53.767782 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e19b10-0546-4237-9af3-9e27e8124be3 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:53.768245 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e19b10-0546-4237-9af3-9e27e8124be3 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:53.768516 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e19b10-0546-4237-9af3-9e27e8124be3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:53.790148 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-20e19b10-0546-4237-9af3-9e27e8124be3 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:33:53.791312 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-20e19b10-0546-4237-9af3-9e27e8124be3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:53.797085 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-3bb68b67-dbcb-49ce-940b-727b666008eb None None] MFA Rules not processed for user `255e4b6fc026453e97de997ec6ab03f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:33:53.798267 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-3bb68b67-dbcb-49ce-940b-727b666008eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:53.806510 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:33:53.808778 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:33:53.875225 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 845/1677] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:33:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 31 12:33:53.883266 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62fe555-5ac8-435a-b632-e6a9ff176a5a None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:53.883629 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62fe555-5ac8-435a-b632-e6a9ff176a5a None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:53.884068 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62fe555-5ac8-435a-b632-e6a9ff176a5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:53.913642 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-d62fe555-5ac8-435a-b632-e6a9ff176a5a None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:33:53.916278 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-d62fe555-5ac8-435a-b632-e6a9ff176a5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:53.928247 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 833/1678] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:33:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:33:53.929225 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 846/1679] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:33:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 31 12:33:53.935157 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb68b67-dbcb-49ce-940b-727b666008eb None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:53.940672 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62fe555-5ac8-435a-b632-e6a9ff176a5a None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:53.942522 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-d62fe555-5ac8-435a-b632-e6a9ff176a5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:53.954735 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb68b67-dbcb-49ce-940b-727b666008eb None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:53.962392 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0850f23-51fc-4def-a4f4-cde05ad2651f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:53.963642 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0850f23-51fc-4def-a4f4-cde05ad2651f admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:53.963924 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0850f23-51fc-4def-a4f4-cde05ad2651f admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:53.964193 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0850f23-51fc-4def-a4f4-cde05ad2651f admin admin] PATH_INFO: `/v3/domains` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:53.964841 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0850f23-51fc-4def-a4f4-cde05ad2651f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:33:53.965142 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0850f23-51fc-4def-a4f4-cde05ad2651f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:53.970070 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0850f23-51fc-4def-a4f4-cde05ad2651f admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:53.987128 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 834/1680] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Oct 31 12:33:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 31 12:33:53.991849 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0850f23-51fc-4def-a4f4-cde05ad2651f admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:54.005422 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0850f23-51fc-4def-a4f4-cde05ad2651f admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:54.012164 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d86a007-b009-4d99-a4c8-60bb9df1e6c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:54.013174 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d86a007-b009-4d99-a4c8-60bb9df1e6c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:54.013671 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d86a007-b009-4d99-a4c8-60bb9df1e6c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:54.014087 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d86a007-b009-4d99-a4c8-60bb9df1e6c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:54.014870 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d86a007-b009-4d99-a4c8-60bb9df1e6c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:54.017103 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d86a007-b009-4d99-a4c8-60bb9df1e6c8 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:54.020076 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62fe555-5ac8-435a-b632-e6a9ff176a5a None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:54.030793 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d52fe2-9cdf-42f0-8afd-644e0eed5141 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d6d7ea077d49e8a66c6c6762499702', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:54.034601 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d52fe2-9cdf-42f0-8afd-644e0eed5141 service cinder] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:54.035136 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d52fe2-9cdf-42f0-8afd-644e0eed5141 service cinder] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:54.035896 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d52fe2-9cdf-42f0-8afd-644e0eed5141 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:54.051555 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d52fe2-9cdf-42f0-8afd-644e0eed5141 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:54.053099 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d52fe2-9cdf-42f0-8afd-644e0eed5141 service cinder] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:54.054742 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 835/1681] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:33:53 2022] POST /identity/v3/projects => generated 369 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:33:54.058855 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d86a007-b009-4d99-a4c8-60bb9df1e6c8 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:54.063331 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 847/1682] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:33:53 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 31 12:33:54.088453 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d52fe2-9cdf-42f0-8afd-644e0eed5141 service cinder] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:54.090166 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-33d52fe2-9cdf-42f0-8afd-644e0eed5141 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:54.106010 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d52fe2-9cdf-42f0-8afd-644e0eed5141 service cinder] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:54.141758 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d86a007-b009-4d99-a4c8-60bb9df1e6c8 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:54.147026 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fca5e0-d644-48f9-bddd-54c56acd1e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:54.147706 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fca5e0-d644-48f9-bddd-54c56acd1e17 admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:54.147986 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fca5e0-d644-48f9-bddd-54c56acd1e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:54.148251 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fca5e0-d644-48f9-bddd-54c56acd1e17 admin admin] PATH_INFO: `/v3/users` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:54.148876 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fca5e0-d644-48f9-bddd-54c56acd1e17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:54.150829 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fca5e0-d644-48f9-bddd-54c56acd1e17 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:54.186106 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b513e60-bfaa-4ad7-b1c3-7cccb1140e52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:54.187464 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b513e60-bfaa-4ad7-b1c3-7cccb1140e52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:54.187952 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b513e60-bfaa-4ad7-b1c3-7cccb1140e52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:54.188823 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b513e60-bfaa-4ad7-b1c3-7cccb1140e52 admin admin] PATH_INFO: `/v3/users/255e4b6fc026453e97de997ec6ab03f7` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:54.191008 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b513e60-bfaa-4ad7-b1c3-7cccb1140e52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=255e4b6fc026453e97de997ec6ab03f7)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:54.192784 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 836/1683] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:33:54 2022] POST /identity/v3/users => generated 285 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:33:54.193728 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fca5e0-d644-48f9-bddd-54c56acd1e17 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:54.194484 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b513e60-bfaa-4ad7-b1c3-7cccb1140e52 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:54.217341 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fca5e0-d644-48f9-bddd-54c56acd1e17 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:54.242077 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87423f4-7096-44af-ad82-56c48dc267b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:54.242808 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87423f4-7096-44af-ad82-56c48dc267b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:54.243511 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87423f4-7096-44af-ad82-56c48dc267b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:54.243785 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87423f4-7096-44af-ad82-56c48dc267b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:54.244523 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87423f4-7096-44af-ad82-56c48dc267b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:54.246399 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87423f4-7096-44af-ad82-56c48dc267b7 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:54.278474 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-5b513e60-bfaa-4ad7-b1c3-7cccb1140e52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '255e4b6fc026453e97de997ec6ab03f7'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:54.278929 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 837/1684] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:33:54 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:33:54.280937 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87423f4-7096-44af-ad82-56c48dc267b7 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:54.288523 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-5b513e60-bfaa-4ad7-b1c3-7cccb1140e52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '255e4b6fc026453e97de997ec6ab03f7'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:54.289330 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-5b513e60-bfaa-4ad7-b1c3-7cccb1140e52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '255e4b6fc026453e97de997ec6ab03f7'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:54.290313 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-5b513e60-bfaa-4ad7-b1c3-7cccb1140e52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '255e4b6fc026453e97de997ec6ab03f7'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:54.293917 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:33:54.296141 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87423f4-7096-44af-ad82-56c48dc267b7 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:54.302172 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9412775-aaad-42ba-94f6-171045261f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:54.302798 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9412775-aaad-42ba-94f6-171045261f99 admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:54.303838 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9412775-aaad-42ba-94f6-171045261f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:54.304196 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9412775-aaad-42ba-94f6-171045261f99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:54.305027 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9412775-aaad-42ba-94f6-171045261f99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:54.306899 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9412775-aaad-42ba-94f6-171045261f99 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:54.316735 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-5b513e60-bfaa-4ad7-b1c3-7cccb1140e52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '255e4b6fc026453e97de997ec6ab03f7'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:54.324235 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 838/1685] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:33:54 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:33:54.329135 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9412775-aaad-42ba-94f6-171045261f99 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:54.331749 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-d9412775-aaad-42ba-94f6-171045261f99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:54.338236 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 848/1686] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:33:54 2022] DELETE /identity/v3/users/255e4b6fc026453e97de997ec6ab03f7 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 31 12:33:54.343291 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828761d9-0790-42ff-87e2-f962c80c6b16 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:54.343654 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828761d9-0790-42ff-87e2-f962c80c6b16 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:54.343993 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828761d9-0790-42ff-87e2-f962c80c6b16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:54.349825 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9412775-aaad-42ba-94f6-171045261f99 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:54.379294 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-828761d9-0790-42ff-87e2-f962c80c6b16 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:33:54.380823 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-828761d9-0790-42ff-87e2-f962c80c6b16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:54.417719 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:33:54.479280 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c741eb3-acdc-43d0-a14b-c45d86ca28f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:54.480234 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c741eb3-acdc-43d0-a14b-c45d86ca28f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:54.480511 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c741eb3-acdc-43d0-a14b-c45d86ca28f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:54.480783 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c741eb3-acdc-43d0-a14b-c45d86ca28f2 admin admin] PATH_INFO: `/v3/projects/dcbc17d01343480b9f9a70e81adf10eb/users/c9bc241b8c3e455f8225867c3bbe6cd2/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:54.514709 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 849/1687] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:33:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 31 12:33:54.522485 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c741eb3-acdc-43d0-a14b-c45d86ca28f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcbc17d01343480b9f9a70e81adf10eb, user_id=c9bc241b8c3e455f8225867c3bbe6cd2, role_id=ca1a5695f8bd499aa0b8cedaeecd2cc6)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:54.525087 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828761d9-0790-42ff-87e2-f962c80c6b16 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:54.528538 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-828761d9-0790-42ff-87e2-f962c80c6b16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:54.532352 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c741eb3-acdc-43d0-a14b-c45d86ca28f2 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:54.582508 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 839/1688] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:33:54 2022] PUT /identity/v3/projects/dcbc17d01343480b9f9a70e81adf10eb/users/c9bc241b8c3e455f8225867c3bbe6cd2/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 31 12:33:54.594162 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e074e82-5bcd-463e-9f95-a366c51493fd None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:54.594541 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e074e82-5bcd-463e-9f95-a366c51493fd None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:54.594887 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e074e82-5bcd-463e-9f95-a366c51493fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:54.650146 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-1e074e82-5bcd-463e-9f95-a366c51493fd None None] MFA Rules not processed for user `c9bc241b8c3e455f8225867c3bbe6cd2`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:33:54.651412 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-1e074e82-5bcd-463e-9f95-a366c51493fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:54.698629 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828761d9-0790-42ff-87e2-f962c80c6b16 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:54.788303 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 840/1689] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:33:54 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:33:54.799621 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e074e82-5bcd-463e-9f95-a366c51493fd None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:54.802922 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-1e074e82-5bcd-463e-9f95-a366c51493fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:54.839070 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8e9b09-3988-40a5-8124-030db989ac94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:54.839792 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8e9b09-3988-40a5-8124-030db989ac94 service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:54.840139 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8e9b09-3988-40a5-8124-030db989ac94 service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:54.840477 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8e9b09-3988-40a5-8124-030db989ac94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:54.972507 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:33:55.005346 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e074e82-5bcd-463e-9f95-a366c51493fd None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:55.010344 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976e416d-abfa-477d-8fc1-6dcfc672c893 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4feaa01c5ed64b65b10d4dece823702c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:55.011064 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976e416d-abfa-477d-8fc1-6dcfc672c893 service glance] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:55.011465 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976e416d-abfa-477d-8fc1-6dcfc672c893 service glance] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:55.011774 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976e416d-abfa-477d-8fc1-6dcfc672c893 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:55.023335 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976e416d-abfa-477d-8fc1-6dcfc672c893 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:55.025179 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976e416d-abfa-477d-8fc1-6dcfc672c893 service glance] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:55.040288 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 841/1690] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:33:54 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 31 12:33:55.046930 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8e9b09-3988-40a5-8124-030db989ac94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:55.049855 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8e9b09-3988-40a5-8124-030db989ac94 service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:55.068254 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 850/1691] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:33:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 544 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:33:55.441405 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976e416d-abfa-477d-8fc1-6dcfc672c893 service glance] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:55.443067 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-976e416d-abfa-477d-8fc1-6dcfc672c893 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:55.469186 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976e416d-abfa-477d-8fc1-6dcfc672c893 service glance] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:55.474558 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a458d485-234b-4357-9539-80c378ee8bb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:55.475229 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a458d485-234b-4357-9539-80c378ee8bb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:55.475516 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a458d485-234b-4357-9539-80c378ee8bb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:55.475786 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a458d485-234b-4357-9539-80c378ee8bb2 admin admin] PATH_INFO: `/v3/projects/a59db1f0925a439a910182891aa6ae6b` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:55.478056 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a458d485-234b-4357-9539-80c378ee8bb2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a59db1f0925a439a910182891aa6ae6b)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:55.480040 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a458d485-234b-4357-9539-80c378ee8bb2 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:55.520921 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:33:55.535922 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-a458d485-234b-4357-9539-80c378ee8bb2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a59db1f0925a439a910182891aa6ae6b', 'request_id': 'req-a458d485-234b-4357-9539-80c378ee8bb2'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:55.542498 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-a458d485-234b-4357-9539-80c378ee8bb2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a59db1f0925a439a910182891aa6ae6b', 'request_id': 'req-a458d485-234b-4357-9539-80c378ee8bb2'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:55.550163 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-a458d485-234b-4357-9539-80c378ee8bb2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a59db1f0925a439a910182891aa6ae6b', 'request_id': 'req-a458d485-234b-4357-9539-80c378ee8bb2'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:55.563192 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-a458d485-234b-4357-9539-80c378ee8bb2 admin admin] The token cache is being invalidate because project a59db1f0925a439a910182891aa6ae6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:33:55.563640 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-a458d485-234b-4357-9539-80c378ee8bb2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:55.564191 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-a458d485-234b-4357-9539-80c378ee8bb2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:55.567134 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 842/1692] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:33:55 2022] DELETE /identity/v3/projects/a59db1f0925a439a910182891aa6ae6b => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:33:55.572300 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bf82fb-f113-41b2-a882-653fe15121c1 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:55.573236 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bf82fb-f113-41b2-a882-653fe15121c1 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:55.573755 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bf82fb-f113-41b2-a882-653fe15121c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:55.598757 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-21bf82fb-f113-41b2-a882-653fe15121c1 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:33:55.600155 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-21bf82fb-f113-41b2-a882-653fe15121c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:55.694316 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 851/1693] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:33:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:33:55.701950 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3eff34-f7d3-4a4f-b816-f5110cb92361 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:55.702311 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3eff34-f7d3-4a4f-b816-f5110cb92361 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:55.702666 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3eff34-f7d3-4a4f-b816-f5110cb92361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:55.724451 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-5f3eff34-f7d3-4a4f-b816-f5110cb92361 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:33:55.725878 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-5f3eff34-f7d3-4a4f-b816-f5110cb92361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:55.747751 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 843/1694] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:33:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:33:55.752761 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bf82fb-f113-41b2-a882-653fe15121c1 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:55.766918 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bf82fb-f113-41b2-a882-653fe15121c1 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:55.773020 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3d34b1-af34-4c39-9c45-e7c8a5736a43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:55.773697 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3d34b1-af34-4c39-9c45-e7c8a5736a43 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:55.773980 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3d34b1-af34-4c39-9c45-e7c8a5736a43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:55.774254 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3d34b1-af34-4c39-9c45-e7c8a5736a43 admin admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:55.774900 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3d34b1-af34-4c39-9c45-e7c8a5736a43 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:33:55.775227 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3d34b1-af34-4c39-9c45-e7c8a5736a43 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:55.777679 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3d34b1-af34-4c39-9c45-e7c8a5736a43 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:55.791509 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 852/1695] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Oct 31 12:33:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:33:55.796071 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3eff34-f7d3-4a4f-b816-f5110cb92361 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:55.806915 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3eff34-f7d3-4a4f-b816-f5110cb92361 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:55.817392 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a6c89a-d70e-4563-a1f6-4c28fd3d38bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:55.817931 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a6c89a-d70e-4563-a1f6-4c28fd3d38bc admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:55.818203 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a6c89a-d70e-4563-a1f6-4c28fd3d38bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:55.818472 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a6c89a-d70e-4563-a1f6-4c28fd3d38bc admin admin] PATH_INFO: `/v3/projects` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:55.819109 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a6c89a-d70e-4563-a1f6-4c28fd3d38bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:55.821035 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a6c89a-d70e-4563-a1f6-4c28fd3d38bc admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:55.860806 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 844/1696] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:33:55 2022] POST /identity/v3/projects => generated 367 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 31 12:33:55.865454 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3d34b1-af34-4c39-9c45-e7c8a5736a43 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:55.881091 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3d34b1-af34-4c39-9c45-e7c8a5736a43 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:55.963581 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9894222-d050-42d2-af72-1bf484985f7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:55.964206 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9894222-d050-42d2-af72-1bf484985f7b admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:55.964652 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9894222-d050-42d2-af72-1bf484985f7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:55.964926 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9894222-d050-42d2-af72-1bf484985f7b admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:55.965689 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9894222-d050-42d2-af72-1bf484985f7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:55.967853 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9894222-d050-42d2-af72-1bf484985f7b admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:55.992204 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 853/1697] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:33:55 2022] POST /identity/v3/users => generated 284 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:33:55.998267 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a6c89a-d70e-4563-a1f6-4c28fd3d38bc admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:56.014570 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a6c89a-d70e-4563-a1f6-4c28fd3d38bc admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:56.021414 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6958bff-d56b-4a96-90df-76bff7a9bd32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:56.022215 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6958bff-d56b-4a96-90df-76bff7a9bd32 admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:56.022648 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6958bff-d56b-4a96-90df-76bff7a9bd32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:56.023050 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6958bff-d56b-4a96-90df-76bff7a9bd32 admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:56.023852 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6958bff-d56b-4a96-90df-76bff7a9bd32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:56.025723 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6958bff-d56b-4a96-90df-76bff7a9bd32 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:56.044959 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 845/1698] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:33:55 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:33:56.045524 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9894222-d050-42d2-af72-1bf484985f7b admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:56.059939 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9894222-d050-42d2-af72-1bf484985f7b admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:56.066547 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f2ac47-e298-4774-ada3-66daeb9f5ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:56.067313 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f2ac47-e298-4774-ada3-66daeb9f5ee1 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:56.067784 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f2ac47-e298-4774-ada3-66daeb9f5ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:56.068156 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f2ac47-e298-4774-ada3-66daeb9f5ee1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:56.069454 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f2ac47-e298-4774-ada3-66daeb9f5ee1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:56.071353 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f2ac47-e298-4774-ada3-66daeb9f5ee1 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:56.103676 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 854/1699] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:33:56 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 31 12:33:56.108315 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6958bff-d56b-4a96-90df-76bff7a9bd32 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:56.128646 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6958bff-d56b-4a96-90df-76bff7a9bd32 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:56.133138 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05934fd6-2019-466d-b07a-70a3030ffe5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:56.133806 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05934fd6-2019-466d-b07a-70a3030ffe5b admin admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:56.134085 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05934fd6-2019-466d-b07a-70a3030ffe5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:56.134358 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05934fd6-2019-466d-b07a-70a3030ffe5b admin admin] PATH_INFO: `/v3/projects/4fe4270c111747be8d61e4ae993bd7ba/users/1918cb60ccd04e909b78dca9b0166be9/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:56.166398 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05934fd6-2019-466d-b07a-70a3030ffe5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fe4270c111747be8d61e4ae993bd7ba, user_id=1918cb60ccd04e909b78dca9b0166be9, role_id=ca1a5695f8bd499aa0b8cedaeecd2cc6)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:56.170255 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05934fd6-2019-466d-b07a-70a3030ffe5b admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:56.177430 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f2ac47-e298-4774-ada3-66daeb9f5ee1 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:56.179413 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-59f2ac47-e298-4774-ada3-66daeb9f5ee1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:56.198361 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 846/1700] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:33:56 2022] PUT /identity/v3/projects/4fe4270c111747be8d61e4ae993bd7ba/users/1918cb60ccd04e909b78dca9b0166be9/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:33:56.203964 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada5440-7d6f-4a98-bc06-448432a6a172 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:56.204250 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada5440-7d6f-4a98-bc06-448432a6a172 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:56.204515 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada5440-7d6f-4a98-bc06-448432a6a172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:56.210437 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f2ac47-e298-4774-ada3-66daeb9f5ee1 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:56.248267 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-eada5440-7d6f-4a98-bc06-448432a6a172 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:33:56.249638 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-eada5440-7d6f-4a98-bc06-448432a6a172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:56.279176 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:33:56.298194 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2ca52d-6729-4cc2-886d-8f6f45b729bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:56.299029 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2ca52d-6729-4cc2-886d-8f6f45b729bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:56.299459 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2ca52d-6729-4cc2-886d-8f6f45b729bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:56.299737 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2ca52d-6729-4cc2-886d-8f6f45b729bb admin admin] PATH_INFO: `/v3/users/31de3cb320d5493dab572757bdafa39d` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:56.300915 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2ca52d-6729-4cc2-886d-8f6f45b729bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=31de3cb320d5493dab572757bdafa39d)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:56.302949 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2ca52d-6729-4cc2-886d-8f6f45b729bb admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:56.339849 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 847/1701] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:33:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 31 12:33:56.352879 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eada5440-7d6f-4a98-bc06-448432a6a172 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:56.355370 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-eada5440-7d6f-4a98-bc06-448432a6a172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:56.394823 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-0c2ca52d-6729-4cc2-886d-8f6f45b729bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31de3cb320d5493dab572757bdafa39d'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:56.410314 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-0c2ca52d-6729-4cc2-886d-8f6f45b729bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31de3cb320d5493dab572757bdafa39d'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:56.411205 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-0c2ca52d-6729-4cc2-886d-8f6f45b729bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31de3cb320d5493dab572757bdafa39d'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:56.411733 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-0c2ca52d-6729-4cc2-886d-8f6f45b729bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31de3cb320d5493dab572757bdafa39d'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:56.431087 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-0c2ca52d-6729-4cc2-886d-8f6f45b729bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31de3cb320d5493dab572757bdafa39d'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:56.437296 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eada5440-7d6f-4a98-bc06-448432a6a172 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:56.442639 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c89001-a19d-48d3-9a6e-345929c2f6da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:56.444025 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c89001-a19d-48d3-9a6e-345929c2f6da service neutron] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:56.444299 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c89001-a19d-48d3-9a6e-345929c2f6da service neutron] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:56.444566 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c89001-a19d-48d3-9a6e-345929c2f6da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:56.445416 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 855/1702] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:33:56 2022] DELETE /identity/v3/users/31de3cb320d5493dab572757bdafa39d => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:33:56.446435 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-e7c89001-a19d-48d3-9a6e-345929c2f6da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:56.449643 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544e091e-6b6f-41ba-9596-a07ab7559113 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:56.450028 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544e091e-6b6f-41ba-9596-a07ab7559113 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:56.450366 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544e091e-6b6f-41ba-9596-a07ab7559113 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:56.461074 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c89001-a19d-48d3-9a6e-345929c2f6da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:56.463218 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c89001-a19d-48d3-9a6e-345929c2f6da service neutron] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:56.470310 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:33:56.472856 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-544e091e-6b6f-41ba-9596-a07ab7559113 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:33:56.472856 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-544e091e-6b6f-41ba-9596-a07ab7559113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:56.482254 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 848/1703] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:33:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:33:56.542299 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 856/1704] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:33:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:33:56.548995 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c89001-a19d-48d3-9a6e-345929c2f6da service neutron] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:56.550427 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-e7c89001-a19d-48d3-9a6e-345929c2f6da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:56.561430 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c89001-a19d-48d3-9a6e-345929c2f6da service neutron] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:56.631091 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622634bd-bb87-4f0d-a2f5-a5b3acc2f775 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:56.631793 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622634bd-bb87-4f0d-a2f5-a5b3acc2f775 service neutron] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:56.632072 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622634bd-bb87-4f0d-a2f5-a5b3acc2f775 service neutron] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:56.632343 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622634bd-bb87-4f0d-a2f5-a5b3acc2f775 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:56.642317 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622634bd-bb87-4f0d-a2f5-a5b3acc2f775 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:56.644506 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622634bd-bb87-4f0d-a2f5-a5b3acc2f775 service neutron] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:56.657787 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 849/1705] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:33:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:33:56.984981 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544e091e-6b6f-41ba-9596-a07ab7559113 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:56.988658 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-544e091e-6b6f-41ba-9596-a07ab7559113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:57.012091 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544e091e-6b6f-41ba-9596-a07ab7559113 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:57.018665 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b92120a-00f4-4f05-bb88-4381d12cc082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:57.020157 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b92120a-00f4-4f05-bb88-4381d12cc082 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:57.030023 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b92120a-00f4-4f05-bb88-4381d12cc082 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:57.030656 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b92120a-00f4-4f05-bb88-4381d12cc082 admin admin] PATH_INFO: `/v3/projects/63d191c3102a4e7b8ac74cbea9f3a8db` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:57.032049 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b92120a-00f4-4f05-bb88-4381d12cc082 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63d191c3102a4e7b8ac74cbea9f3a8db)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:57.036624 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b92120a-00f4-4f05-bb88-4381d12cc082 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:57.132089 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-7b92120a-00f4-4f05-bb88-4381d12cc082 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63d191c3102a4e7b8ac74cbea9f3a8db', 'request_id': 'req-7b92120a-00f4-4f05-bb88-4381d12cc082'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:57.140115 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-7b92120a-00f4-4f05-bb88-4381d12cc082 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63d191c3102a4e7b8ac74cbea9f3a8db', 'request_id': 'req-7b92120a-00f4-4f05-bb88-4381d12cc082'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:57.159182 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-7b92120a-00f4-4f05-bb88-4381d12cc082 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63d191c3102a4e7b8ac74cbea9f3a8db', 'request_id': 'req-7b92120a-00f4-4f05-bb88-4381d12cc082'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:57.161725 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:33:57.185541 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-7b92120a-00f4-4f05-bb88-4381d12cc082 admin admin] The token cache is being invalidate because project 63d191c3102a4e7b8ac74cbea9f3a8db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:33:57.185839 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-7b92120a-00f4-4f05-bb88-4381d12cc082 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:57.188230 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-7b92120a-00f4-4f05-bb88-4381d12cc082 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:57.191439 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 857/1706] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:33:56 2022] DELETE /identity/v3/projects/63d191c3102a4e7b8ac74cbea9f3a8db => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:33:57.195270 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622634bd-bb87-4f0d-a2f5-a5b3acc2f775 service neutron] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:57.208263 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622634bd-bb87-4f0d-a2f5-a5b3acc2f775 service neutron] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:57.296661 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654a2d4a-5e3a-44ef-a01d-03453bfd0f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:57.297306 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654a2d4a-5e3a-44ef-a01d-03453bfd0f6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:57.297725 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654a2d4a-5e3a-44ef-a01d-03453bfd0f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:57.298004 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654a2d4a-5e3a-44ef-a01d-03453bfd0f6d admin admin] PATH_INFO: `/v3/users/bb9e3c9c9f234cdb939bb21d12e2237c` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:57.299736 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654a2d4a-5e3a-44ef-a01d-03453bfd0f6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb9e3c9c9f234cdb939bb21d12e2237c)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:57.301645 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654a2d4a-5e3a-44ef-a01d-03453bfd0f6d admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:57.360310 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-654a2d4a-5e3a-44ef-a01d-03453bfd0f6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb9e3c9c9f234cdb939bb21d12e2237c'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:57.370205 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-654a2d4a-5e3a-44ef-a01d-03453bfd0f6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb9e3c9c9f234cdb939bb21d12e2237c'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:57.370895 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-654a2d4a-5e3a-44ef-a01d-03453bfd0f6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb9e3c9c9f234cdb939bb21d12e2237c'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:57.388913 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-654a2d4a-5e3a-44ef-a01d-03453bfd0f6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb9e3c9c9f234cdb939bb21d12e2237c'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:57.398113 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-654a2d4a-5e3a-44ef-a01d-03453bfd0f6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb9e3c9c9f234cdb939bb21d12e2237c'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:57.406130 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 850/1707] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:33:57 2022] DELETE /identity/v3/users/bb9e3c9c9f234cdb939bb21d12e2237c => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:33:58.342272 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b92120a-00f4-4f05-bb88-4381d12cc082 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:58.343754 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-7b92120a-00f4-4f05-bb88-4381d12cc082 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:58.366357 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b92120a-00f4-4f05-bb88-4381d12cc082 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:58.422030 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d34c67c-c9df-43bb-a6f4-c2dc874d6ed1 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:58.422030 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d34c67c-c9df-43bb-a6f4-c2dc874d6ed1 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:58.422030 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d34c67c-c9df-43bb-a6f4-c2dc874d6ed1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:58.495157 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-1d34c67c-c9df-43bb-a6f4-c2dc874d6ed1 None None] MFA Rules not processed for user `1918cb60ccd04e909b78dca9b0166be9`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:33:58.495157 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-1d34c67c-c9df-43bb-a6f4-c2dc874d6ed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:58.514630 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:33:58.656647 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73352962-63c5-413f-951b-ebd6718e7cde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:58.656647 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73352962-63c5-413f-951b-ebd6718e7cde admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:58.656647 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73352962-63c5-413f-951b-ebd6718e7cde admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:58.656971 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73352962-63c5-413f-951b-ebd6718e7cde admin admin] PATH_INFO: `/v3/projects/fcd5c7c179ca41db9e897ec17c515e72` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:58.659692 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73352962-63c5-413f-951b-ebd6718e7cde admin admin] RBAC: Authorizing `identity:delete_project(project_id=fcd5c7c179ca41db9e897ec17c515e72)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:58.661686 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73352962-63c5-413f-951b-ebd6718e7cde admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:58.665171 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 851/1708] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:33:58 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 31 12:33:58.677200 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d34c67c-c9df-43bb-a6f4-c2dc874d6ed1 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:58.680569 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-1d34c67c-c9df-43bb-a6f4-c2dc874d6ed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:58.775692 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-73352962-63c5-413f-951b-ebd6718e7cde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fcd5c7c179ca41db9e897ec17c515e72', 'request_id': 'req-73352962-63c5-413f-951b-ebd6718e7cde'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:58.791887 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-73352962-63c5-413f-951b-ebd6718e7cde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fcd5c7c179ca41db9e897ec17c515e72', 'request_id': 'req-73352962-63c5-413f-951b-ebd6718e7cde'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:58.801057 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d34c67c-c9df-43bb-a6f4-c2dc874d6ed1 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:58.809118 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-73352962-63c5-413f-951b-ebd6718e7cde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fcd5c7c179ca41db9e897ec17c515e72', 'request_id': 'req-73352962-63c5-413f-951b-ebd6718e7cde'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:58.827383 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-73352962-63c5-413f-951b-ebd6718e7cde admin admin] The token cache is being invalidate because project fcd5c7c179ca41db9e897ec17c515e72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:33:58.828154 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-73352962-63c5-413f-951b-ebd6718e7cde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:58.829156 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-73352962-63c5-413f-951b-ebd6718e7cde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:58.836171 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 858/1709] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:33:58 2022] DELETE /identity/v3/projects/fcd5c7c179ca41db9e897ec17c515e72 => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:33:58.840880 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73352962-63c5-413f-951b-ebd6718e7cde admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:58.848271 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-73352962-63c5-413f-951b-ebd6718e7cde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:58.927969 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a16d7b8-1611-4533-b14a-dce4d5c16cba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d6d7ea077d49e8a66c6c6762499702', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:58.928541 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a16d7b8-1611-4533-b14a-dce4d5c16cba service cinder] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:58.928934 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a16d7b8-1611-4533-b14a-dce4d5c16cba service cinder] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:58.929342 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a16d7b8-1611-4533-b14a-dce4d5c16cba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:58.931365 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-9a16d7b8-1611-4533-b14a-dce4d5c16cba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:58.991554 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73352962-63c5-413f-951b-ebd6718e7cde admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:59.005117 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc860d2-e2e6-4302-8eed-8a50caba1c77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:59.006789 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc860d2-e2e6-4302-8eed-8a50caba1c77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:59.009696 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc860d2-e2e6-4302-8eed-8a50caba1c77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:59.009867 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc860d2-e2e6-4302-8eed-8a50caba1c77 admin admin] PATH_INFO: `/v3/users/170b5ff87168436693104d2478e9c0ac` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:59.013446 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc860d2-e2e6-4302-8eed-8a50caba1c77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=170b5ff87168436693104d2478e9c0ac)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:59.021756 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc860d2-e2e6-4302-8eed-8a50caba1c77 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:59.058877 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:33:59.110707 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a16d7b8-1611-4533-b14a-dce4d5c16cba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:59.113867 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a16d7b8-1611-4533-b14a-dce4d5c16cba service cinder] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:59.122257 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-6dc860d2-e2e6-4302-8eed-8a50caba1c77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '170b5ff87168436693104d2478e9c0ac'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:59.136858 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-6dc860d2-e2e6-4302-8eed-8a50caba1c77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '170b5ff87168436693104d2478e9c0ac'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:59.138441 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-6dc860d2-e2e6-4302-8eed-8a50caba1c77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '170b5ff87168436693104d2478e9c0ac'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:59.140141 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-6dc860d2-e2e6-4302-8eed-8a50caba1c77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '170b5ff87168436693104d2478e9c0ac'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:59.141360 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 852/1710] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:33:58 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 31 12:33:59.188052 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-6dc860d2-e2e6-4302-8eed-8a50caba1c77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '170b5ff87168436693104d2478e9c0ac'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:59.220143 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 859/1711] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:33:58 2022] DELETE /identity/v3/users/170b5ff87168436693104d2478e9c0ac => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:33:59.675553 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a16d7b8-1611-4533-b14a-dce4d5c16cba service cinder] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:33:59.697531 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a16d7b8-1611-4533-b14a-dce4d5c16cba service cinder] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:33:59.718149 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:33:59.760978 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3324da41-e10b-4d86-b687-928847d2b8aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:33:59.762025 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3324da41-e10b-4d86-b687-928847d2b8aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:59.762310 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3324da41-e10b-4d86-b687-928847d2b8aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:59.762746 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3324da41-e10b-4d86-b687-928847d2b8aa admin admin] PATH_INFO: `/v3/projects/8cae26d8ae7f412292629591048c675f` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:59.763922 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3324da41-e10b-4d86-b687-928847d2b8aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=8cae26d8ae7f412292629591048c675f)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:33:59.765854 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3324da41-e10b-4d86-b687-928847d2b8aa admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:33:59.821838 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-3324da41-e10b-4d86-b687-928847d2b8aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8cae26d8ae7f412292629591048c675f', 'request_id': 'req-3324da41-e10b-4d86-b687-928847d2b8aa'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:59.827631 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-3324da41-e10b-4d86-b687-928847d2b8aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8cae26d8ae7f412292629591048c675f', 'request_id': 'req-3324da41-e10b-4d86-b687-928847d2b8aa'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:59.835940 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-3324da41-e10b-4d86-b687-928847d2b8aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8cae26d8ae7f412292629591048c675f', 'request_id': 'req-3324da41-e10b-4d86-b687-928847d2b8aa'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:59.847973 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-3324da41-e10b-4d86-b687-928847d2b8aa admin admin] The token cache is being invalidate because project 8cae26d8ae7f412292629591048c675f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:33:59.848285 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-3324da41-e10b-4d86-b687-928847d2b8aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:59.848918 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-3324da41-e10b-4d86-b687-928847d2b8aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:33:59.850425 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 853/1712] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:33:59 2022] DELETE /identity/v3/projects/8cae26d8ae7f412292629591048c675f => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:33:59.855608 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bd633e-3624-4759-ad34-937dc75da914 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:59.856108 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bd633e-3624-4759-ad34-937dc75da914 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:59.856197 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bd633e-3624-4759-ad34-937dc75da914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:33:59.876518 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-e6bd633e-3624-4759-ad34-937dc75da914 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:33:59.878326 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-e6bd633e-3624-4759-ad34-937dc75da914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:33:59.967307 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 860/1713] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:33:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:33:59.971916 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f7f3b1-ae2a-41ac-b459-e4a012a0c3c1 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:33:59.974341 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f7f3b1-ae2a-41ac-b459-e4a012a0c3c1 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:33:59.974584 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f7f3b1-ae2a-41ac-b459-e4a012a0c3c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:00.008413 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-f2f7f3b1-ae2a-41ac-b459-e4a012a0c3c1 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:34:00.009763 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-f2f7f3b1-ae2a-41ac-b459-e4a012a0c3c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:34:00.029800 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 854/1714] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:33:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:34:00.034137 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bd633e-3624-4759-ad34-937dc75da914 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:00.046614 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bd633e-3624-4759-ad34-937dc75da914 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:00.052213 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a0bc74-5803-470f-aab9-aac92105e2c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:00.052947 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a0bc74-5803-470f-aab9-aac92105e2c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:00.053326 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a0bc74-5803-470f-aab9-aac92105e2c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:00.053670 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a0bc74-5803-470f-aab9-aac92105e2c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:00.054420 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a0bc74-5803-470f-aab9-aac92105e2c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:34:00.054820 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a0bc74-5803-470f-aab9-aac92105e2c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:00.056569 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a0bc74-5803-470f-aab9-aac92105e2c2 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:00.070628 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 861/1715] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Oct 31 12:34:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:34:00.075076 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f7f3b1-ae2a-41ac-b459-e4a012a0c3c1 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:00.088252 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f7f3b1-ae2a-41ac-b459-e4a012a0c3c1 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:00.093324 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c47ab0-3d0a-4afe-8c1c-75f3b912e924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:00.094043 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c47ab0-3d0a-4afe-8c1c-75f3b912e924 admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:00.094250 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c47ab0-3d0a-4afe-8c1c-75f3b912e924 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:00.094464 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c47ab0-3d0a-4afe-8c1c-75f3b912e924 admin admin] PATH_INFO: `/v3/projects` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:00.095092 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c47ab0-3d0a-4afe-8c1c-75f3b912e924 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:00.097110 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c47ab0-3d0a-4afe-8c1c-75f3b912e924 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:00.136961 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 855/1716] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:34:00 2022] POST /identity/v3/projects => generated 371 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:34:00.141972 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a0bc74-5803-470f-aab9-aac92105e2c2 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:00.153675 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a0bc74-5803-470f-aab9-aac92105e2c2 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:00.214451 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7f67c9-3a78-4fa4-a2c6-8bef9210f2e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:00.214987 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7f67c9-3a78-4fa4-a2c6-8bef9210f2e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:00.215376 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7f67c9-3a78-4fa4-a2c6-8bef9210f2e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:00.215642 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7f67c9-3a78-4fa4-a2c6-8bef9210f2e7 admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:00.216297 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7f67c9-3a78-4fa4-a2c6-8bef9210f2e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:00.218262 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7f67c9-3a78-4fa4-a2c6-8bef9210f2e7 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:00.243082 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 862/1717] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:34:00 2022] POST /identity/v3/users => generated 286 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:34:00.247338 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c47ab0-3d0a-4afe-8c1c-75f3b912e924 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:00.258196 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c47ab0-3d0a-4afe-8c1c-75f3b912e924 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:00.262886 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec13b265-3f4a-4821-87ca-24eb5d9e1262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:00.263531 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec13b265-3f4a-4821-87ca-24eb5d9e1262 admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:00.263814 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec13b265-3f4a-4821-87ca-24eb5d9e1262 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:00.264110 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec13b265-3f4a-4821-87ca-24eb5d9e1262 admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:00.264771 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec13b265-3f4a-4821-87ca-24eb5d9e1262 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:00.266351 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec13b265-3f4a-4821-87ca-24eb5d9e1262 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:00.278316 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 856/1718] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:34:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 31 12:34:00.282213 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7f67c9-3a78-4fa4-a2c6-8bef9210f2e7 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:00.293157 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7f67c9-3a78-4fa4-a2c6-8bef9210f2e7 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:00.297945 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4b3399-ffc3-4184-9092-26045afcde97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:00.298651 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4b3399-ffc3-4184-9092-26045afcde97 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:00.300119 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4b3399-ffc3-4184-9092-26045afcde97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:00.300441 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4b3399-ffc3-4184-9092-26045afcde97 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:00.301848 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4b3399-ffc3-4184-9092-26045afcde97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:00.302870 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4b3399-ffc3-4184-9092-26045afcde97 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:00.315205 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 863/1719] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:34:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:34:00.319545 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec13b265-3f4a-4821-87ca-24eb5d9e1262 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:00.330117 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec13b265-3f4a-4821-87ca-24eb5d9e1262 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:00.335009 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950fd6c7-6e48-4f3a-996d-d742d6cfcb10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:00.335722 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950fd6c7-6e48-4f3a-996d-d742d6cfcb10 admin admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:00.336003 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950fd6c7-6e48-4f3a-996d-d742d6cfcb10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:00.336272 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950fd6c7-6e48-4f3a-996d-d742d6cfcb10 admin admin] PATH_INFO: `/v3/projects/b14b98f006574deabc030c2e93c680c4/users/825059319a204b6da5bc7004b3a09361/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:00.351916 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950fd6c7-6e48-4f3a-996d-d742d6cfcb10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b14b98f006574deabc030c2e93c680c4, user_id=825059319a204b6da5bc7004b3a09361, role_id=ca1a5695f8bd499aa0b8cedaeecd2cc6)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:00.354545 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950fd6c7-6e48-4f3a-996d-d742d6cfcb10 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:00.372912 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 857/1720] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:34:00 2022] PUT /identity/v3/projects/b14b98f006574deabc030c2e93c680c4/users/825059319a204b6da5bc7004b3a09361/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:34:00.378486 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0badb53-6a77-41ec-ad41-001a3c6fd7fe None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:00.378742 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0badb53-6a77-41ec-ad41-001a3c6fd7fe None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:00.379074 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0badb53-6a77-41ec-ad41-001a3c6fd7fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:00.387473 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:34:00.399550 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-d0badb53-6a77-41ec-ad41-001a3c6fd7fe None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:34:00.401360 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-d0badb53-6a77-41ec-ad41-001a3c6fd7fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:34:00.461677 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 864/1721] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:34:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:34:00.471956 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950fd6c7-6e48-4f3a-996d-d742d6cfcb10 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:00.473774 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-950fd6c7-6e48-4f3a-996d-d742d6cfcb10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:34:00.528435 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:34:00.534063 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950fd6c7-6e48-4f3a-996d-d742d6cfcb10 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:00.543190 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d172434-373b-4567-9679-9b59529169e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:00.544012 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d172434-373b-4567-9679-9b59529169e1 service neutron] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:00.544455 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d172434-373b-4567-9679-9b59529169e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:00.544773 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d172434-373b-4567-9679-9b59529169e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:00.556965 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d172434-373b-4567-9679-9b59529169e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:00.558376 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d172434-373b-4567-9679-9b59529169e1 service neutron] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:00.567875 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 858/1722] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:34:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 31 12:34:00.645981 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912618b7-48f4-4b48-9574-e2bb2b42521e None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:00.646307 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912618b7-48f4-4b48-9574-e2bb2b42521e None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:00.646539 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912618b7-48f4-4b48-9574-e2bb2b42521e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:00.667466 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-912618b7-48f4-4b48-9574-e2bb2b42521e None None] MFA Rules not processed for user `1918cb60ccd04e909b78dca9b0166be9`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:34:00.668745 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-912618b7-48f4-4b48-9574-e2bb2b42521e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:34:00.741024 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 865/1723] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:34:00 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:34:09.343372 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b6d2c0-6970-4793-a7c9-f5c9bb410565 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:09.343993 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b6d2c0-6970-4793-a7c9-f5c9bb410565 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:09.343993 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b6d2c0-6970-4793-a7c9-f5c9bb410565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:09.380538 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-a7b6d2c0-6970-4793-a7c9-f5c9bb410565 None None] MFA Rules not processed for user `825059319a204b6da5bc7004b3a09361`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:34:09.382032 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-a7b6d2c0-6970-4793-a7c9-f5c9bb410565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:34:09.477537 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 859/1724] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:34:09 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:34:09.478705 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912618b7-48f4-4b48-9574-e2bb2b42521e None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:09.489505 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912618b7-48f4-4b48-9574-e2bb2b42521e None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:09.495701 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66afcf19-14a5-42b2-9394-84b2d4b63b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:09.496434 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66afcf19-14a5-42b2-9394-84b2d4b63b97 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:09.496611 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66afcf19-14a5-42b2-9394-84b2d4b63b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:09.496865 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66afcf19-14a5-42b2-9394-84b2d4b63b97 admin admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:09.497648 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66afcf19-14a5-42b2-9394-84b2d4b63b97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:09.499588 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66afcf19-14a5-42b2-9394-84b2d4b63b97 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:09.551513 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 866/1725] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:34:09 2022] POST /identity/v3/projects => generated 373 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 31 12:34:09.555562 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b6d2c0-6970-4793-a7c9-f5c9bb410565 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:09.570359 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b6d2c0-6970-4793-a7c9-f5c9bb410565 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:09.695556 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0891d86d-a4dc-4293-8a36-c89a6d73375a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:09.696173 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0891d86d-a4dc-4293-8a36-c89a6d73375a admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:09.696525 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0891d86d-a4dc-4293-8a36-c89a6d73375a admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:09.696806 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0891d86d-a4dc-4293-8a36-c89a6d73375a admin admin] PATH_INFO: `/v3/users` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:09.697442 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0891d86d-a4dc-4293-8a36-c89a6d73375a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:09.699794 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0891d86d-a4dc-4293-8a36-c89a6d73375a admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:09.739240 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 860/1726] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:34:09 2022] POST /identity/v3/users => generated 293 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:34:09.741770 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66afcf19-14a5-42b2-9394-84b2d4b63b97 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:09.758442 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66afcf19-14a5-42b2-9394-84b2d4b63b97 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:09.779068 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ae96ef-2d38-4dba-87f6-4a226b6a271b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:09.779863 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ae96ef-2d38-4dba-87f6-4a226b6a271b admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:09.780141 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ae96ef-2d38-4dba-87f6-4a226b6a271b admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:09.780413 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ae96ef-2d38-4dba-87f6-4a226b6a271b admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:09.781067 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ae96ef-2d38-4dba-87f6-4a226b6a271b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:09.783002 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ae96ef-2d38-4dba-87f6-4a226b6a271b admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:09.824490 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 867/1727] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:34:09 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:34:09.825505 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0891d86d-a4dc-4293-8a36-c89a6d73375a admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:09.843312 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0891d86d-a4dc-4293-8a36-c89a6d73375a admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:09.854242 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6358bf6f-f546-4886-a161-f67e6ddcee58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:09.856680 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6358bf6f-f546-4886-a161-f67e6ddcee58 admin admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:09.856990 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6358bf6f-f546-4886-a161-f67e6ddcee58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:09.857267 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6358bf6f-f546-4886-a161-f67e6ddcee58 admin admin] PATH_INFO: `/v3/projects/656dea667748432aadfffc81fb3a8cb3/users/aa1cbcd62cb04c6a9258474764721df0/roles/176c07b4fe3b4537bdfeb4be21b379c5` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:09.921124 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6358bf6f-f546-4886-a161-f67e6ddcee58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=656dea667748432aadfffc81fb3a8cb3, user_id=aa1cbcd62cb04c6a9258474764721df0, role_id=176c07b4fe3b4537bdfeb4be21b379c5)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:09.929184 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6358bf6f-f546-4886-a161-f67e6ddcee58 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:09.956075 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 861/1728] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:34:09 2022] PUT /identity/v3/projects/656dea667748432aadfffc81fb3a8cb3/users/aa1cbcd62cb04c6a9258474764721df0/roles/176c07b4fe3b4537bdfeb4be21b379c5 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 31 12:34:09.960576 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ae96ef-2d38-4dba-87f6-4a226b6a271b admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:09.976834 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ae96ef-2d38-4dba-87f6-4a226b6a271b admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:10.094357 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf3463a-8f0b-4210-b23e-692ac2ca8b0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:10.095622 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf3463a-8f0b-4210-b23e-692ac2ca8b0a admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:10.097094 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf3463a-8f0b-4210-b23e-692ac2ca8b0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:10.097920 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf3463a-8f0b-4210-b23e-692ac2ca8b0a admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:10.099719 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf3463a-8f0b-4210-b23e-692ac2ca8b0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:10.103747 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf3463a-8f0b-4210-b23e-692ac2ca8b0a admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:10.147492 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 868/1729] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:34:09 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:34:10.148566 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6358bf6f-f546-4886-a161-f67e6ddcee58 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:10.159829 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6358bf6f-f546-4886-a161-f67e6ddcee58 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:10.195955 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d64f7f-02e2-49b5-bb94-2e174458b3c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:10.196660 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d64f7f-02e2-49b5-bb94-2e174458b3c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:10.196936 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d64f7f-02e2-49b5-bb94-2e174458b3c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:10.197207 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d64f7f-02e2-49b5-bb94-2e174458b3c8 admin admin] PATH_INFO: `/v3/projects/656dea667748432aadfffc81fb3a8cb3/users/aa1cbcd62cb04c6a9258474764721df0/roles/176c07b4fe3b4537bdfeb4be21b379c5` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:10.201503 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d64f7f-02e2-49b5-bb94-2e174458b3c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=656dea667748432aadfffc81fb3a8cb3, user_id=aa1cbcd62cb04c6a9258474764721df0, role_id=176c07b4fe3b4537bdfeb4be21b379c5)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:10.204210 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d64f7f-02e2-49b5-bb94-2e174458b3c8 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:10.250125 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 862/1730] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:34:10 2022] PUT /identity/v3/projects/656dea667748432aadfffc81fb3a8cb3/users/aa1cbcd62cb04c6a9258474764721df0/roles/176c07b4fe3b4537bdfeb4be21b379c5 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:34:12.363152 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a322ea2f-b9d6-4c20-bec5-50cc7a73c897 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:12.363709 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a322ea2f-b9d6-4c20-bec5-50cc7a73c897 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:12.363853 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a322ea2f-b9d6-4c20-bec5-50cc7a73c897 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:12.365994 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 869/1731] 10.222.0.15 () {54 vars in 921 bytes} [Mon Oct 31 12:34:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:34:12.391893 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1044047-a6c2-40a5-a5ef-00520453c4fa None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:12.393988 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1044047-a6c2-40a5-a5ef-00520453c4fa None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:12.393988 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1044047-a6c2-40a5-a5ef-00520453c4fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:12.431666 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-d1044047-a6c2-40a5-a5ef-00520453c4fa None None] MFA Rules not processed for user `9e7777495d7a406c8c50b1ad52e47263`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:34:12.433331 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-d1044047-a6c2-40a5-a5ef-00520453c4fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:34:12.490187 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:34:12.552587 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a322ea2f-b9d6-4c20-bec5-50cc7a73c897 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:12.554878 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-a322ea2f-b9d6-4c20-bec5-50cc7a73c897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:34:12.578230 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:34:12.585915 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 863/1732] 10.222.0.15 () {58 vars in 1018 bytes} [Mon Oct 31 12:34:12 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:34:12.607581 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1044047-a6c2-40a5-a5ef-00520453c4fa None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:12.609097 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-d1044047-a6c2-40a5-a5ef-00520453c4fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:34:12.634087 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1044047-a6c2-40a5-a5ef-00520453c4fa None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:12.643198 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e6b59d-de63-44be-a9ff-33f5b14e716c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e7777495d7a406c8c50b1ad52e47263', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:12.643750 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e6b59d-de63-44be-a9ff-33f5b14e716c service placement] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:12.644023 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e6b59d-de63-44be-a9ff-33f5b14e716c service placement] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:12.644288 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e6b59d-de63-44be-a9ff-33f5b14e716c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:12.657289 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e6b59d-de63-44be-a9ff-33f5b14e716c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:12.660925 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e6b59d-de63-44be-a9ff-33f5b14e716c service placement] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:12.685759 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 864/1733] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:34:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:34:12.700278 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a322ea2f-b9d6-4c20-bec5-50cc7a73c897 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:12.709852 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b3e642-ea84-4888-bc38-b129d73ec587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:12.710607 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b3e642-ea84-4888-bc38-b129d73ec587 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:12.710939 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b3e642-ea84-4888-bc38-b129d73ec587 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:12.711448 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b3e642-ea84-4888-bc38-b129d73ec587 admin admin] PATH_INFO: `/v3/users/70d59ee6f94848308665a32099cf5255` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:12.713562 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b3e642-ea84-4888-bc38-b129d73ec587 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70d59ee6f94848308665a32099cf5255)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:12.715581 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b3e642-ea84-4888-bc38-b129d73ec587 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:12.790655 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-11b3e642-ea84-4888-bc38-b129d73ec587 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70d59ee6f94848308665a32099cf5255'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:12.803804 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-11b3e642-ea84-4888-bc38-b129d73ec587 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70d59ee6f94848308665a32099cf5255'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:12.805176 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-11b3e642-ea84-4888-bc38-b129d73ec587 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70d59ee6f94848308665a32099cf5255'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:12.806194 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-11b3e642-ea84-4888-bc38-b129d73ec587 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70d59ee6f94848308665a32099cf5255'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:12.828077 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-11b3e642-ea84-4888-bc38-b129d73ec587 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70d59ee6f94848308665a32099cf5255'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:12.854645 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 870/1734] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:34:12 2022] DELETE /identity/v3/users/70d59ee6f94848308665a32099cf5255 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 31 12:34:13.143831 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e6b59d-de63-44be-a9ff-33f5b14e716c service placement] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:13.161627 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e6b59d-de63-44be-a9ff-33f5b14e716c service placement] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:13.281314 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0083f0c-0601-48df-85ee-41c98cee9c36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:13.283560 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0083f0c-0601-48df-85ee-41c98cee9c36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:13.284049 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0083f0c-0601-48df-85ee-41c98cee9c36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:13.299699 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0083f0c-0601-48df-85ee-41c98cee9c36 admin admin] PATH_INFO: `/v3/projects/a4d5cd8851dc4216b4e3e035a32decad` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:13.303844 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0083f0c-0601-48df-85ee-41c98cee9c36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4d5cd8851dc4216b4e3e035a32decad)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:13.308752 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0083f0c-0601-48df-85ee-41c98cee9c36 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:13.407223 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-f0083f0c-0601-48df-85ee-41c98cee9c36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4d5cd8851dc4216b4e3e035a32decad', 'request_id': 'req-f0083f0c-0601-48df-85ee-41c98cee9c36'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:13.417405 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-f0083f0c-0601-48df-85ee-41c98cee9c36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4d5cd8851dc4216b4e3e035a32decad', 'request_id': 'req-f0083f0c-0601-48df-85ee-41c98cee9c36'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:13.429869 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-f0083f0c-0601-48df-85ee-41c98cee9c36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4d5cd8851dc4216b4e3e035a32decad', 'request_id': 'req-f0083f0c-0601-48df-85ee-41c98cee9c36'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:13.460151 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-f0083f0c-0601-48df-85ee-41c98cee9c36 admin admin] The token cache is being invalidate because project a4d5cd8851dc4216b4e3e035a32decad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:34:13.460631 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-f0083f0c-0601-48df-85ee-41c98cee9c36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:13.461632 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-f0083f0c-0601-48df-85ee-41c98cee9c36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:13.464157 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 865/1735] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:34:13 2022] DELETE /identity/v3/projects/a4d5cd8851dc4216b4e3e035a32decad => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:34:13.471638 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691e18d8-1b81-4c60-9d32-6f6174067082 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:13.472187 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691e18d8-1b81-4c60-9d32-6f6174067082 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:13.472187 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691e18d8-1b81-4c60-9d32-6f6174067082 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:13.508606 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-691e18d8-1b81-4c60-9d32-6f6174067082 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:34:13.511889 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-691e18d8-1b81-4c60-9d32-6f6174067082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:34:13.725210 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 871/1736] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:34:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:34:13.733548 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56592f23-21a4-4d6c-838d-312ce0354890 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:13.734013 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56592f23-21a4-4d6c-838d-312ce0354890 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:13.734437 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56592f23-21a4-4d6c-838d-312ce0354890 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:13.791826 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-56592f23-21a4-4d6c-838d-312ce0354890 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:34:13.793067 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-56592f23-21a4-4d6c-838d-312ce0354890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:34:13.815405 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 866/1737] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:34:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:34:13.816975 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691e18d8-1b81-4c60-9d32-6f6174067082 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:13.834718 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691e18d8-1b81-4c60-9d32-6f6174067082 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:13.842720 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3314ca20-0700-4abd-9c43-a9e45e0a7cb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:13.843714 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3314ca20-0700-4abd-9c43-a9e45e0a7cb8 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:13.844062 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3314ca20-0700-4abd-9c43-a9e45e0a7cb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:13.847878 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3314ca20-0700-4abd-9c43-a9e45e0a7cb8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:13.849127 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3314ca20-0700-4abd-9c43-a9e45e0a7cb8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:34:13.849646 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3314ca20-0700-4abd-9c43-a9e45e0a7cb8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:13.853017 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3314ca20-0700-4abd-9c43-a9e45e0a7cb8 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:13.876517 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 872/1738] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Oct 31 12:34:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:34:13.878286 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56592f23-21a4-4d6c-838d-312ce0354890 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:13.893369 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56592f23-21a4-4d6c-838d-312ce0354890 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:13.934198 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea6c007-e1c4-4a35-9b69-8f72b3510ce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:13.939146 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea6c007-e1c4-4a35-9b69-8f72b3510ce7 admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:13.939952 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea6c007-e1c4-4a35-9b69-8f72b3510ce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:13.940428 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea6c007-e1c4-4a35-9b69-8f72b3510ce7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:13.941297 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea6c007-e1c4-4a35-9b69-8f72b3510ce7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:13.943904 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea6c007-e1c4-4a35-9b69-8f72b3510ce7 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:14.008690 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 867/1739] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:34:13 2022] POST /identity/v3/projects => generated 383 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:34:14.009194 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3314ca20-0700-4abd-9c43-a9e45e0a7cb8 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:14.023729 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3314ca20-0700-4abd-9c43-a9e45e0a7cb8 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:14.061431 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:34:14.129923 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c37614-7e65-43e1-8535-d7c021cbf8da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:14.130747 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c37614-7e65-43e1-8535-d7c021cbf8da admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:14.131230 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c37614-7e65-43e1-8535-d7c021cbf8da admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:14.131664 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c37614-7e65-43e1-8535-d7c021cbf8da admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:14.132584 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c37614-7e65-43e1-8535-d7c021cbf8da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:14.135609 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c37614-7e65-43e1-8535-d7c021cbf8da admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:14.212964 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 873/1740] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:34:14 2022] POST /identity/v3/users => generated 292 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:34:14.217173 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea6c007-e1c4-4a35-9b69-8f72b3510ce7 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:14.231393 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea6c007-e1c4-4a35-9b69-8f72b3510ce7 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:14.253699 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089da732-1c36-41ae-9ab1-d04377ab5459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:14.254499 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089da732-1c36-41ae-9ab1-d04377ab5459 admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:14.254777 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089da732-1c36-41ae-9ab1-d04377ab5459 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:14.256596 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089da732-1c36-41ae-9ab1-d04377ab5459 admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:14.257288 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089da732-1c36-41ae-9ab1-d04377ab5459 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:14.259238 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089da732-1c36-41ae-9ab1-d04377ab5459 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:14.273918 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 868/1741] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:34:14 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:34:14.278972 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c37614-7e65-43e1-8535-d7c021cbf8da admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:14.292288 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c37614-7e65-43e1-8535-d7c021cbf8da admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:14.298651 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c1ca1b-acd8-47eb-b627-9653822b451b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:14.299522 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c1ca1b-acd8-47eb-b627-9653822b451b admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:14.299850 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c1ca1b-acd8-47eb-b627-9653822b451b admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:14.300292 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c1ca1b-acd8-47eb-b627-9653822b451b admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:14.301048 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c1ca1b-acd8-47eb-b627-9653822b451b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:14.302985 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c1ca1b-acd8-47eb-b627-9653822b451b admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:14.316244 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 874/1742] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:34:14 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:34:14.320945 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089da732-1c36-41ae-9ab1-d04377ab5459 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:14.332124 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089da732-1c36-41ae-9ab1-d04377ab5459 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:14.337832 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175845b2-32d0-4c80-bdda-1a44dba2510c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:14.338485 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175845b2-32d0-4c80-bdda-1a44dba2510c admin admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:14.338763 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175845b2-32d0-4c80-bdda-1a44dba2510c admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:14.339094 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175845b2-32d0-4c80-bdda-1a44dba2510c admin admin] PATH_INFO: `/v3/projects/44205e3c7d864276bb80d9b5a59d266b/users/03b31c6585714f0187a06fd433be18ad/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:14.341527 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:34:14.359534 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175845b2-32d0-4c80-bdda-1a44dba2510c admin admin] RBAC: Authorizing `identity:create_grant(project_id=44205e3c7d864276bb80d9b5a59d266b, user_id=03b31c6585714f0187a06fd433be18ad, role_id=ca1a5695f8bd499aa0b8cedaeecd2cc6)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:14.362763 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175845b2-32d0-4c80-bdda-1a44dba2510c admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:14.378367 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 869/1743] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:34:14 2022] PUT /identity/v3/projects/44205e3c7d864276bb80d9b5a59d266b/users/03b31c6585714f0187a06fd433be18ad/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:34:14.389029 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41b0df5-1eb8-49d9-a591-340275b8b551 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:14.389312 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41b0df5-1eb8-49d9-a591-340275b8b551 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:14.389728 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41b0df5-1eb8-49d9-a591-340275b8b551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:14.422110 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-b41b0df5-1eb8-49d9-a591-340275b8b551 None None] MFA Rules not processed for user `03b31c6585714f0187a06fd433be18ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:34:14.423894 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-b41b0df5-1eb8-49d9-a591-340275b8b551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:34:14.511372 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 875/1744] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:34:14 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 31 12:34:14.521520 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175845b2-32d0-4c80-bdda-1a44dba2510c admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:14.523749 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-175845b2-32d0-4c80-bdda-1a44dba2510c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:34:14.635956 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175845b2-32d0-4c80-bdda-1a44dba2510c admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:14.642631 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46355d60-8a47-40e7-95ec-d2ed8044dfe7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99889b1dab0b471d9c77887cffefc652', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:14.643480 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46355d60-8a47-40e7-95ec-d2ed8044dfe7 service nova] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:14.643836 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46355d60-8a47-40e7-95ec-d2ed8044dfe7 service nova] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:14.644182 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46355d60-8a47-40e7-95ec-d2ed8044dfe7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:14.655582 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46355d60-8a47-40e7-95ec-d2ed8044dfe7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:14.660880 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46355d60-8a47-40e7-95ec-d2ed8044dfe7 service nova] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:14.691973 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 870/1745] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:34:14 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:34:16.944748 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e602b1-9de0-44e3-80e2-3e2197e4d64b None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:16.945297 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e602b1-9de0-44e3-80e2-3e2197e4d64b None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:16.945297 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e602b1-9de0-44e3-80e2-3e2197e4d64b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:16.982125 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-f1e602b1-9de0-44e3-80e2-3e2197e4d64b None None] MFA Rules not processed for user `aa1cbcd62cb04c6a9258474764721df0`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:34:16.982125 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-f1e602b1-9de0-44e3-80e2-3e2197e4d64b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:34:17.074296 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 876/1746] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:34:16 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:34:17.087513 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46355d60-8a47-40e7-95ec-d2ed8044dfe7 service nova] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:17.089046 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-46355d60-8a47-40e7-95ec-d2ed8044dfe7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:34:17.104146 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46355d60-8a47-40e7-95ec-d2ed8044dfe7 service nova] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:17.109721 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6d900f-67c8-4356-8f43-9467ff663b0a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99889b1dab0b471d9c77887cffefc652', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:17.110328 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6d900f-67c8-4356-8f43-9467ff663b0a service nova] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:17.111192 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6d900f-67c8-4356-8f43-9467ff663b0a service nova] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:17.111477 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6d900f-67c8-4356-8f43-9467ff663b0a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:17.113255 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-0f6d900f-67c8-4356-8f43-9467ff663b0a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:34:17.178598 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6d900f-67c8-4356-8f43-9467ff663b0a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:17.180317 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6d900f-67c8-4356-8f43-9467ff663b0a service nova] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:17.190306 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 871/1747] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:34:17 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:34:18.281149 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90ef3e6-2d9f-4cbb-89d2-c416a0f8fdd0 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:18.281647 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90ef3e6-2d9f-4cbb-89d2-c416a0f8fdd0 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:18.281737 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90ef3e6-2d9f-4cbb-89d2-c416a0f8fdd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:18.302512 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-d90ef3e6-2d9f-4cbb-89d2-c416a0f8fdd0 None None] MFA Rules not processed for user `825059319a204b6da5bc7004b3a09361`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:34:18.303747 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-d90ef3e6-2d9f-4cbb-89d2-c416a0f8fdd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:34:18.310587 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 877/1748] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:34:18 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:34:27.186209 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=69,keystone:INSERT=1 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:34:28.217651 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2450f19d-0f85-482e-8b9b-9f30b71f120a None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:28.218334 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2450f19d-0f85-482e-8b9b-9f30b71f120a None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:28.218334 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2450f19d-0f85-482e-8b9b-9f30b71f120a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:28.248228 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-2450f19d-0f85-482e-8b9b-9f30b71f120a None None] MFA Rules not processed for user `1918cb60ccd04e909b78dca9b0166be9`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:34:28.249625 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-2450f19d-0f85-482e-8b9b-9f30b71f120a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:34:28.295311 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=94,keystone:INSERT=3 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:34:28.313762 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 872/1749] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:34:28 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:34:28.837515 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90ef3e6-2d9f-4cbb-89d2-c416a0f8fdd0 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:28.905826 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90ef3e6-2d9f-4cbb-89d2-c416a0f8fdd0 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:28.910982 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3162c2b5-d0d4-427c-b9aa-e8fad9174a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:28.911655 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3162c2b5-d0d4-427c-b9aa-e8fad9174a61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:28.911932 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3162c2b5-d0d4-427c-b9aa-e8fad9174a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:28.912205 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3162c2b5-d0d4-427c-b9aa-e8fad9174a61 admin admin] PATH_INFO: `/v3/users/03b31c6585714f0187a06fd433be18ad` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:28.913330 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3162c2b5-d0d4-427c-b9aa-e8fad9174a61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=03b31c6585714f0187a06fd433be18ad)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:28.915260 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3162c2b5-d0d4-427c-b9aa-e8fad9174a61 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:28.981653 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-3162c2b5-d0d4-427c-b9aa-e8fad9174a61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03b31c6585714f0187a06fd433be18ad'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:28.989736 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-3162c2b5-d0d4-427c-b9aa-e8fad9174a61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03b31c6585714f0187a06fd433be18ad'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:28.990358 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-3162c2b5-d0d4-427c-b9aa-e8fad9174a61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03b31c6585714f0187a06fd433be18ad'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:28.990982 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-3162c2b5-d0d4-427c-b9aa-e8fad9174a61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03b31c6585714f0187a06fd433be18ad'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:29.004577 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-3162c2b5-d0d4-427c-b9aa-e8fad9174a61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03b31c6585714f0187a06fd433be18ad'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:29.021720 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 878/1750] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:34:28 2022] DELETE /identity/v3/users/03b31c6585714f0187a06fd433be18ad => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:34:29.026195 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c19ceb-42a0-4a78-8022-d7cc5ca65924 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:29.026481 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c19ceb-42a0-4a78-8022-d7cc5ca65924 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:29.026754 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c19ceb-42a0-4a78-8022-d7cc5ca65924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:29.048748 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-31c19ceb-42a0-4a78-8022-d7cc5ca65924 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:34:29.050051 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-31c19ceb-42a0-4a78-8022-d7cc5ca65924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:34:29.113358 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 873/1751] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:34:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:34:29.122965 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3162c2b5-d0d4-427c-b9aa-e8fad9174a61 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:29.124474 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-3162c2b5-d0d4-427c-b9aa-e8fad9174a61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:34:29.189610 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3162c2b5-d0d4-427c-b9aa-e8fad9174a61 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:29.193511 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e453d8c0-f63b-4ee1-8e33-7d1fe8272a87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:29.194139 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e453d8c0-f63b-4ee1-8e33-7d1fe8272a87 service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:29.194421 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e453d8c0-f63b-4ee1-8e33-7d1fe8272a87 service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:29.194691 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e453d8c0-f63b-4ee1-8e33-7d1fe8272a87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:29.203875 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e453d8c0-f63b-4ee1-8e33-7d1fe8272a87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:29.205229 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e453d8c0-f63b-4ee1-8e33-7d1fe8272a87 service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:29.216024 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 879/1752] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:34:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:34:29.614473 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c19ceb-42a0-4a78-8022-d7cc5ca65924 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:29.616035 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-31c19ceb-42a0-4a78-8022-d7cc5ca65924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:34:29.631260 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c19ceb-42a0-4a78-8022-d7cc5ca65924 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:29.636859 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda0907d-0a1d-44b9-b13e-59e759142053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:29.637905 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda0907d-0a1d-44b9-b13e-59e759142053 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:29.638197 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda0907d-0a1d-44b9-b13e-59e759142053 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:29.638473 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda0907d-0a1d-44b9-b13e-59e759142053 admin admin] PATH_INFO: `/v3/projects/44205e3c7d864276bb80d9b5a59d266b` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:29.639631 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda0907d-0a1d-44b9-b13e-59e759142053 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44205e3c7d864276bb80d9b5a59d266b)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:29.641826 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda0907d-0a1d-44b9-b13e-59e759142053 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:29.694151 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-dda0907d-0a1d-44b9-b13e-59e759142053 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44205e3c7d864276bb80d9b5a59d266b', 'request_id': 'req-dda0907d-0a1d-44b9-b13e-59e759142053'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:29.699925 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-dda0907d-0a1d-44b9-b13e-59e759142053 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44205e3c7d864276bb80d9b5a59d266b', 'request_id': 'req-dda0907d-0a1d-44b9-b13e-59e759142053'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:29.707522 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-dda0907d-0a1d-44b9-b13e-59e759142053 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44205e3c7d864276bb80d9b5a59d266b', 'request_id': 'req-dda0907d-0a1d-44b9-b13e-59e759142053'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:29.720272 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-dda0907d-0a1d-44b9-b13e-59e759142053 admin admin] The token cache is being invalidate because project 44205e3c7d864276bb80d9b5a59d266b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:34:29.720640 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-dda0907d-0a1d-44b9-b13e-59e759142053 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:29.721339 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-dda0907d-0a1d-44b9-b13e-59e759142053 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:29.722842 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 874/1753] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:34:29 2022] DELETE /identity/v3/projects/44205e3c7d864276bb80d9b5a59d266b => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:34:29.728221 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598ff7e6-7725-484a-bd8c-ff95e45bb249 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:29.728680 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598ff7e6-7725-484a-bd8c-ff95e45bb249 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:29.728771 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598ff7e6-7725-484a-bd8c-ff95e45bb249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:29.749056 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-598ff7e6-7725-484a-bd8c-ff95e45bb249 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:34:29.750370 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-598ff7e6-7725-484a-bd8c-ff95e45bb249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:34:29.775119 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:34:29.821726 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 880/1754] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:34:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:34:29.830888 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c634f55-9c8d-4664-9fec-7f72e0d3d0c2 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:29.831128 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c634f55-9c8d-4664-9fec-7f72e0d3d0c2 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:29.831464 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c634f55-9c8d-4664-9fec-7f72e0d3d0c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:29.850919 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-8c634f55-9c8d-4664-9fec-7f72e0d3d0c2 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:34:29.852159 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-8c634f55-9c8d-4664-9fec-7f72e0d3d0c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:34:29.859963 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 875/1755] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:34:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:34:29.864673 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598ff7e6-7725-484a-bd8c-ff95e45bb249 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:29.875671 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598ff7e6-7725-484a-bd8c-ff95e45bb249 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:29.881742 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406fa522-8f0c-4bab-8e6a-0f61189141ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:29.882362 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406fa522-8f0c-4bab-8e6a-0f61189141ad admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:29.882640 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406fa522-8f0c-4bab-8e6a-0f61189141ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:29.882910 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406fa522-8f0c-4bab-8e6a-0f61189141ad admin admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:29.883592 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406fa522-8f0c-4bab-8e6a-0f61189141ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:34:29.883895 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406fa522-8f0c-4bab-8e6a-0f61189141ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:29.885479 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406fa522-8f0c-4bab-8e6a-0f61189141ad admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:29.899079 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 881/1756] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Oct 31 12:34:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 31 12:34:29.902509 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c634f55-9c8d-4664-9fec-7f72e0d3d0c2 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:29.911874 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c634f55-9c8d-4664-9fec-7f72e0d3d0c2 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:29.916533 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7923701-8791-44b7-bc8a-ca31afabd86e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:29.917112 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7923701-8791-44b7-bc8a-ca31afabd86e admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:29.917389 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7923701-8791-44b7-bc8a-ca31afabd86e admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:29.917659 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7923701-8791-44b7-bc8a-ca31afabd86e admin admin] PATH_INFO: `/v3/projects` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:29.918285 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7923701-8791-44b7-bc8a-ca31afabd86e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:29.920218 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7923701-8791-44b7-bc8a-ca31afabd86e admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:29.942245 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:34:29.952216 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 876/1757] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:34:29 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:34:29.953339 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406fa522-8f0c-4bab-8e6a-0f61189141ad admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:29.963216 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406fa522-8f0c-4bab-8e6a-0f61189141ad admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:30.024190 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14234808-edc1-4813-8b4a-11823326dda4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:30.024804 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14234808-edc1-4813-8b4a-11823326dda4 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:30.025084 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14234808-edc1-4813-8b4a-11823326dda4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:30.025387 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14234808-edc1-4813-8b4a-11823326dda4 admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:30.026009 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14234808-edc1-4813-8b4a-11823326dda4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:30.028001 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14234808-edc1-4813-8b4a-11823326dda4 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:30.053099 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 882/1758] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:34:29 2022] POST /identity/v3/users => generated 292 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:34:30.056771 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7923701-8791-44b7-bc8a-ca31afabd86e admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:30.068901 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7923701-8791-44b7-bc8a-ca31afabd86e admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:30.078600 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bb5087-2eca-4055-85ea-6e7a053ad3a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:30.079360 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bb5087-2eca-4055-85ea-6e7a053ad3a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:30.079644 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bb5087-2eca-4055-85ea-6e7a053ad3a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:30.079915 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bb5087-2eca-4055-85ea-6e7a053ad3a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:30.080556 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bb5087-2eca-4055-85ea-6e7a053ad3a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:30.082262 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bb5087-2eca-4055-85ea-6e7a053ad3a9 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:30.095760 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 877/1759] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:34:30 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 31 12:34:30.096947 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14234808-edc1-4813-8b4a-11823326dda4 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:30.107566 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14234808-edc1-4813-8b4a-11823326dda4 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:30.113364 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17af8795-b55d-46e0-939b-4de033e9171e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:30.113967 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17af8795-b55d-46e0-939b-4de033e9171e admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:30.114242 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17af8795-b55d-46e0-939b-4de033e9171e admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:30.114510 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17af8795-b55d-46e0-939b-4de033e9171e admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:30.115140 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17af8795-b55d-46e0-939b-4de033e9171e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:30.116734 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17af8795-b55d-46e0-939b-4de033e9171e admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:30.128514 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 883/1760] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:34:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:34:30.133497 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bb5087-2eca-4055-85ea-6e7a053ad3a9 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:30.143818 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bb5087-2eca-4055-85ea-6e7a053ad3a9 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:30.152068 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074408e1-5a87-4ab7-9b1a-f364ec8a21f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:30.152785 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074408e1-5a87-4ab7-9b1a-f364ec8a21f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:30.153061 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074408e1-5a87-4ab7-9b1a-f364ec8a21f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:30.153334 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074408e1-5a87-4ab7-9b1a-f364ec8a21f7 admin admin] PATH_INFO: `/v3/projects/a94eb7c546204d99bcd054b0d7e0cd04/users/2aac2e24c00d4fd2bd4aed51b74b30b3/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:30.176105 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074408e1-5a87-4ab7-9b1a-f364ec8a21f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a94eb7c546204d99bcd054b0d7e0cd04, user_id=2aac2e24c00d4fd2bd4aed51b74b30b3, role_id=ca1a5695f8bd499aa0b8cedaeecd2cc6)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:30.179278 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074408e1-5a87-4ab7-9b1a-f364ec8a21f7 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:30.194759 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 878/1761] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:34:30 2022] PUT /identity/v3/projects/a94eb7c546204d99bcd054b0d7e0cd04/users/2aac2e24c00d4fd2bd4aed51b74b30b3/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:34:30.205459 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea656f4a-f2fd-47f8-a30c-f0e4b84e95aa None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:30.205741 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea656f4a-f2fd-47f8-a30c-f0e4b84e95aa None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:30.206011 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea656f4a-f2fd-47f8-a30c-f0e4b84e95aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:30.234691 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-ea656f4a-f2fd-47f8-a30c-f0e4b84e95aa None None] MFA Rules not processed for user `2aac2e24c00d4fd2bd4aed51b74b30b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:34:30.236541 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-ea656f4a-f2fd-47f8-a30c-f0e4b84e95aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:34:30.315081 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 884/1762] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:34:30 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:34:30.319636 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074408e1-5a87-4ab7-9b1a-f364ec8a21f7 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:30.400805 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074408e1-5a87-4ab7-9b1a-f364ec8a21f7 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:30.405729 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40af842-df9b-4418-a00c-b931ce18208b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:30.406337 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40af842-df9b-4418-a00c-b931ce18208b admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:30.406617 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40af842-df9b-4418-a00c-b931ce18208b admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:30.406944 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40af842-df9b-4418-a00c-b931ce18208b admin admin] PATH_INFO: `/v3/projects` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:30.408194 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40af842-df9b-4418-a00c-b931ce18208b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:30.412667 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40af842-df9b-4418-a00c-b931ce18208b admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:30.448442 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 879/1763] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:34:30 2022] POST /identity/v3/projects => generated 381 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 31 12:34:30.449595 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea656f4a-f2fd-47f8-a30c-f0e4b84e95aa None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:30.461424 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea656f4a-f2fd-47f8-a30c-f0e4b84e95aa None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:30.474434 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:34:30.526597 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f87ab3-4d05-402a-b193-b5440226a2cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:30.527276 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f87ab3-4d05-402a-b193-b5440226a2cb admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:30.527555 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f87ab3-4d05-402a-b193-b5440226a2cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:30.527826 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f87ab3-4d05-402a-b193-b5440226a2cb admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:30.528460 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f87ab3-4d05-402a-b193-b5440226a2cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:30.530359 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f87ab3-4d05-402a-b193-b5440226a2cb admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:30.558127 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 885/1764] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:34:30 2022] POST /identity/v3/users => generated 297 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 31 12:34:30.562219 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40af842-df9b-4418-a00c-b931ce18208b admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:30.576276 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40af842-df9b-4418-a00c-b931ce18208b admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:30.581666 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6e1186-1b40-4b85-9e32-59d2e1487cde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:30.582969 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6e1186-1b40-4b85-9e32-59d2e1487cde admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:30.582969 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6e1186-1b40-4b85-9e32-59d2e1487cde admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:30.583581 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6e1186-1b40-4b85-9e32-59d2e1487cde admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:30.584292 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6e1186-1b40-4b85-9e32-59d2e1487cde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:30.586244 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6e1186-1b40-4b85-9e32-59d2e1487cde admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:30.599024 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 880/1765] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:34:30 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:34:30.603879 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f87ab3-4d05-402a-b193-b5440226a2cb admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:30.616494 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f87ab3-4d05-402a-b193-b5440226a2cb admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:30.621641 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bfb0f7-f39a-4341-81d9-07c0feb193b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:30.622303 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bfb0f7-f39a-4341-81d9-07c0feb193b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:30.622580 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bfb0f7-f39a-4341-81d9-07c0feb193b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:30.622861 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bfb0f7-f39a-4341-81d9-07c0feb193b3 admin admin] PATH_INFO: `/v3/projects/692870f0e1ac467c809e6597fdb992e1/users/68cb861cbd0c41ccbc2157980efa87b3/roles/176c07b4fe3b4537bdfeb4be21b379c5` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:30.642507 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bfb0f7-f39a-4341-81d9-07c0feb193b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=692870f0e1ac467c809e6597fdb992e1, user_id=68cb861cbd0c41ccbc2157980efa87b3, role_id=176c07b4fe3b4537bdfeb4be21b379c5)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:30.646014 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bfb0f7-f39a-4341-81d9-07c0feb193b3 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:30.659887 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 886/1766] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:34:30 2022] PUT /identity/v3/projects/692870f0e1ac467c809e6597fdb992e1/users/68cb861cbd0c41ccbc2157980efa87b3/roles/176c07b4fe3b4537bdfeb4be21b379c5 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:34:30.663911 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6e1186-1b40-4b85-9e32-59d2e1487cde admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:30.674504 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6e1186-1b40-4b85-9e32-59d2e1487cde admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:30.743189 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c5acf3-b4da-4158-8588-72c2e573258b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:30.743863 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c5acf3-b4da-4158-8588-72c2e573258b admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:30.744159 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c5acf3-b4da-4158-8588-72c2e573258b admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:30.744426 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c5acf3-b4da-4158-8588-72c2e573258b admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:30.745062 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c5acf3-b4da-4158-8588-72c2e573258b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:30.746627 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c5acf3-b4da-4158-8588-72c2e573258b admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:30.757190 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 881/1767] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:34:30 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:34:30.762682 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bfb0f7-f39a-4341-81d9-07c0feb193b3 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:30.776614 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bfb0f7-f39a-4341-81d9-07c0feb193b3 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:30.782170 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d55c31-9f05-43b8-9f63-d5a5e326227f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:30.783692 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d55c31-9f05-43b8-9f63-d5a5e326227f admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:30.783972 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d55c31-9f05-43b8-9f63-d5a5e326227f admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:30.784242 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d55c31-9f05-43b8-9f63-d5a5e326227f admin admin] PATH_INFO: `/v3/projects/692870f0e1ac467c809e6597fdb992e1/users/68cb861cbd0c41ccbc2157980efa87b3/roles/176c07b4fe3b4537bdfeb4be21b379c5` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:30.786175 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d55c31-9f05-43b8-9f63-d5a5e326227f admin admin] RBAC: Authorizing `identity:create_grant(project_id=692870f0e1ac467c809e6597fdb992e1, user_id=68cb861cbd0c41ccbc2157980efa87b3, role_id=176c07b4fe3b4537bdfeb4be21b379c5)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:30.788954 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d55c31-9f05-43b8-9f63-d5a5e326227f admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:30.800098 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 887/1768] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:34:30 2022] PUT /identity/v3/projects/692870f0e1ac467c809e6597fdb992e1/users/68cb861cbd0c41ccbc2157980efa87b3/roles/176c07b4fe3b4537bdfeb4be21b379c5 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:34:30.811082 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a330c0-4392-419d-aa4f-4e19b1c5d303 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:30.811454 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a330c0-4392-419d-aa4f-4e19b1c5d303 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:30.811752 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a330c0-4392-419d-aa4f-4e19b1c5d303 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:30.848471 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-b9a330c0-4392-419d-aa4f-4e19b1c5d303 None None] MFA Rules not processed for user `68cb861cbd0c41ccbc2157980efa87b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:34:30.849696 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-b9a330c0-4392-419d-aa4f-4e19b1c5d303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:34:30.874796 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:34:30.928872 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 882/1769] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:34:30 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:34:30.942420 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d55c31-9f05-43b8-9f63-d5a5e326227f admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:30.944419 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-c5d55c31-9f05-43b8-9f63-d5a5e326227f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:34:31.008121 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d55c31-9f05-43b8-9f63-d5a5e326227f admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:31.011955 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c129c4-e454-4598-be6d-294a4ba2d218 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d6d7ea077d49e8a66c6c6762499702', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:31.012583 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c129c4-e454-4598-be6d-294a4ba2d218 service cinder] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:31.012862 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c129c4-e454-4598-be6d-294a4ba2d218 service cinder] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:31.013132 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c129c4-e454-4598-be6d-294a4ba2d218 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:31.025015 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c129c4-e454-4598-be6d-294a4ba2d218 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:31.026456 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c129c4-e454-4598-be6d-294a4ba2d218 service cinder] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:31.039137 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 888/1770] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:34:30 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 31 12:34:31.183129 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a330c0-4392-419d-aa4f-4e19b1c5d303 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:31.193869 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a330c0-4392-419d-aa4f-4e19b1c5d303 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:31.201722 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e9bc96-499c-4789-a3bb-e8364e66dfe1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d6d7ea077d49e8a66c6c6762499702', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:31.202348 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e9bc96-499c-4789-a3bb-e8364e66dfe1 service cinder] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:31.202627 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e9bc96-499c-4789-a3bb-e8364e66dfe1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:31.202896 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e9bc96-499c-4789-a3bb-e8364e66dfe1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:31.271427 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e9bc96-499c-4789-a3bb-e8364e66dfe1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:31.273092 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e9bc96-499c-4789-a3bb-e8364e66dfe1 service cinder] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:31.283975 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 883/1771] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:34:31 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:34:33.947041 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c129c4-e454-4598-be6d-294a4ba2d218 service cinder] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:33.967315 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e9bc96-499c-4789-a3bb-e8364e66dfe1 service cinder] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:33.968777 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-37e9bc96-499c-4789-a3bb-e8364e66dfe1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:34:34.047482 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c129c4-e454-4598-be6d-294a4ba2d218 service cinder] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:34.054546 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaee43b-3e43-408c-8ef0-bc7963515ca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:34.058589 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaee43b-3e43-408c-8ef0-bc7963515ca8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:34.058589 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaee43b-3e43-408c-8ef0-bc7963515ca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:34.058589 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaee43b-3e43-408c-8ef0-bc7963515ca8 admin admin] PATH_INFO: `/v3/users/2aac2e24c00d4fd2bd4aed51b74b30b3` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:34.058589 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaee43b-3e43-408c-8ef0-bc7963515ca8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2aac2e24c00d4fd2bd4aed51b74b30b3)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:34.059674 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaee43b-3e43-408c-8ef0-bc7963515ca8 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:34.067259 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e9bc96-499c-4789-a3bb-e8364e66dfe1 service cinder] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:34.074679 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dab012-3f64-4e26-9ba5-bc97fddd5ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:34.075498 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dab012-3f64-4e26-9ba5-bc97fddd5ca5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:34.075881 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dab012-3f64-4e26-9ba5-bc97fddd5ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:34.076252 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dab012-3f64-4e26-9ba5-bc97fddd5ca5 admin admin] PATH_INFO: `/v3/users/c9bc241b8c3e455f8225867c3bbe6cd2` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:34.076671 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:34:34.077649 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dab012-3f64-4e26-9ba5-bc97fddd5ca5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9bc241b8c3e455f8225867c3bbe6cd2)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:34.079757 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dab012-3f64-4e26-9ba5-bc97fddd5ca5 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:34.122876 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-caaee43b-3e43-408c-8ef0-bc7963515ca8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2aac2e24c00d4fd2bd4aed51b74b30b3'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:34.146506 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-caaee43b-3e43-408c-8ef0-bc7963515ca8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2aac2e24c00d4fd2bd4aed51b74b30b3'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:34.147310 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-caaee43b-3e43-408c-8ef0-bc7963515ca8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2aac2e24c00d4fd2bd4aed51b74b30b3'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:34.148164 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-caaee43b-3e43-408c-8ef0-bc7963515ca8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2aac2e24c00d4fd2bd4aed51b74b30b3'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:34.150230 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-61dab012-3f64-4e26-9ba5-bc97fddd5ca5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9bc241b8c3e455f8225867c3bbe6cd2'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:34.162720 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-61dab012-3f64-4e26-9ba5-bc97fddd5ca5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9bc241b8c3e455f8225867c3bbe6cd2'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:34.164687 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-61dab012-3f64-4e26-9ba5-bc97fddd5ca5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9bc241b8c3e455f8225867c3bbe6cd2'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:34.167706 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-caaee43b-3e43-408c-8ef0-bc7963515ca8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2aac2e24c00d4fd2bd4aed51b74b30b3'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:34.174888 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:34:34.185800 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-61dab012-3f64-4e26-9ba5-bc97fddd5ca5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9bc241b8c3e455f8225867c3bbe6cd2'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:34.191865 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 889/1772] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:34:33 2022] DELETE /identity/v3/users/2aac2e24c00d4fd2bd4aed51b74b30b3 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:34:34.200549 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6890e3-22e9-4a0c-a752-003639240755 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:34.201172 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6890e3-22e9-4a0c-a752-003639240755 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:34.201368 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-61dab012-3f64-4e26-9ba5-bc97fddd5ca5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9bc241b8c3e455f8225867c3bbe6cd2'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:34.201857 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6890e3-22e9-4a0c-a752-003639240755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:34.214163 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 884/1773] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:34:33 2022] DELETE /identity/v3/users/c9bc241b8c3e455f8225867c3bbe6cd2 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:34:34.219380 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc07a7d1-6e7c-48c2-9550-6e85c86d2059 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:34.220271 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc07a7d1-6e7c-48c2-9550-6e85c86d2059 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:34.220776 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc07a7d1-6e7c-48c2-9550-6e85c86d2059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:34.231148 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-ef6890e3-22e9-4a0c-a752-003639240755 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:34:34.232693 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-ef6890e3-22e9-4a0c-a752-003639240755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:34:34.245835 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-bc07a7d1-6e7c-48c2-9550-6e85c86d2059 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:34:34.247249 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-bc07a7d1-6e7c-48c2-9550-6e85c86d2059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:34:34.317565 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 890/1774] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:34:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:34:34.322253 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 885/1775] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:34:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:34:34.332748 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6890e3-22e9-4a0c-a752-003639240755 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:34.334648 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-ef6890e3-22e9-4a0c-a752-003639240755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:34:34.339765 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc07a7d1-6e7c-48c2-9550-6e85c86d2059 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:34.341679 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-bc07a7d1-6e7c-48c2-9550-6e85c86d2059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:34:34.411508 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6890e3-22e9-4a0c-a752-003639240755 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:34.415501 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18a19c1-3fef-4a7b-b92a-d42416ac3bb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:34.416238 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18a19c1-3fef-4a7b-b92a-d42416ac3bb1 service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:34.416384 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18a19c1-3fef-4a7b-b92a-d42416ac3bb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:34.416648 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18a19c1-3fef-4a7b-b92a-d42416ac3bb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:34.421268 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc07a7d1-6e7c-48c2-9550-6e85c86d2059 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:34.425830 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5523a57-c565-42fe-9020-2b01e5fba931 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:34.426515 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5523a57-c565-42fe-9020-2b01e5fba931 service neutron] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:34.426860 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5523a57-c565-42fe-9020-2b01e5fba931 service neutron] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:34.427231 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5523a57-c565-42fe-9020-2b01e5fba931 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:34.428957 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18a19c1-3fef-4a7b-b92a-d42416ac3bb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:34.430445 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18a19c1-3fef-4a7b-b92a-d42416ac3bb1 service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:34.439729 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5523a57-c565-42fe-9020-2b01e5fba931 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:34.441273 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 891/1776] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:34:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:34:34.441622 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5523a57-c565-42fe-9020-2b01e5fba931 service neutron] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:34.457169 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 886/1777] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:34:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:34:34.955517 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18a19c1-3fef-4a7b-b92a-d42416ac3bb1 service neutron] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:34.957168 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-f18a19c1-3fef-4a7b-b92a-d42416ac3bb1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:34:34.971189 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5523a57-c565-42fe-9020-2b01e5fba931 service neutron] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:34.971961 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18a19c1-3fef-4a7b-b92a-d42416ac3bb1 service neutron] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:34.972934 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-e5523a57-c565-42fe-9020-2b01e5fba931 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:34:34.976697 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987860f5-4096-4b53-b0df-e07b832c2a49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:34.977336 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987860f5-4096-4b53-b0df-e07b832c2a49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:34.977615 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987860f5-4096-4b53-b0df-e07b832c2a49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:34.977885 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987860f5-4096-4b53-b0df-e07b832c2a49 admin admin] PATH_INFO: `/v3/projects/a94eb7c546204d99bcd054b0d7e0cd04` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:34.979024 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987860f5-4096-4b53-b0df-e07b832c2a49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a94eb7c546204d99bcd054b0d7e0cd04)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:34.980959 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987860f5-4096-4b53-b0df-e07b832c2a49 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:34.990934 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5523a57-c565-42fe-9020-2b01e5fba931 service neutron] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:35.003910 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3220e9-6696-4752-bf4b-2794a66b3487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:35.004573 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3220e9-6696-4752-bf4b-2794a66b3487 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:35.004851 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3220e9-6696-4752-bf4b-2794a66b3487 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:35.005121 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3220e9-6696-4752-bf4b-2794a66b3487 admin admin] PATH_INFO: `/v3/projects/dcbc17d01343480b9f9a70e81adf10eb` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:35.006279 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3220e9-6696-4752-bf4b-2794a66b3487 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dcbc17d01343480b9f9a70e81adf10eb)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:35.008509 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3220e9-6696-4752-bf4b-2794a66b3487 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:35.037703 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:34:35.053089 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-987860f5-4096-4b53-b0df-e07b832c2a49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a94eb7c546204d99bcd054b0d7e0cd04', 'request_id': 'req-987860f5-4096-4b53-b0df-e07b832c2a49'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:35.063486 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-987860f5-4096-4b53-b0df-e07b832c2a49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a94eb7c546204d99bcd054b0d7e0cd04', 'request_id': 'req-987860f5-4096-4b53-b0df-e07b832c2a49'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:35.069512 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-9e3220e9-6696-4752-bf4b-2794a66b3487 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dcbc17d01343480b9f9a70e81adf10eb', 'request_id': 'req-9e3220e9-6696-4752-bf4b-2794a66b3487'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:35.072581 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-987860f5-4096-4b53-b0df-e07b832c2a49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a94eb7c546204d99bcd054b0d7e0cd04', 'request_id': 'req-987860f5-4096-4b53-b0df-e07b832c2a49'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:35.075827 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-9e3220e9-6696-4752-bf4b-2794a66b3487 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dcbc17d01343480b9f9a70e81adf10eb', 'request_id': 'req-9e3220e9-6696-4752-bf4b-2794a66b3487'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:35.083925 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-9e3220e9-6696-4752-bf4b-2794a66b3487 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dcbc17d01343480b9f9a70e81adf10eb', 'request_id': 'req-9e3220e9-6696-4752-bf4b-2794a66b3487'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:35.085121 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-987860f5-4096-4b53-b0df-e07b832c2a49 admin admin] The token cache is being invalidate because project a94eb7c546204d99bcd054b0d7e0cd04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:34:35.085428 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-987860f5-4096-4b53-b0df-e07b832c2a49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:35.086055 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-987860f5-4096-4b53-b0df-e07b832c2a49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:35.087591 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 892/1778] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:34:34 2022] DELETE /identity/v3/projects/a94eb7c546204d99bcd054b0d7e0cd04 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 31 12:34:35.091565 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987860f5-4096-4b53-b0df-e07b832c2a49 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:35.092961 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-987860f5-4096-4b53-b0df-e07b832c2a49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:34:35.099558 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-9e3220e9-6696-4752-bf4b-2794a66b3487 admin admin] The token cache is being invalidate because project dcbc17d01343480b9f9a70e81adf10eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:34:35.099921 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-9e3220e9-6696-4752-bf4b-2794a66b3487 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:35.100641 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-9e3220e9-6696-4752-bf4b-2794a66b3487 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:35.102171 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 887/1779] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:34:34 2022] DELETE /identity/v3/projects/dcbc17d01343480b9f9a70e81adf10eb => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:34:35.106723 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c958e987-9b9f-4875-9a77-bc2a479a6fcb None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:35.107014 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c958e987-9b9f-4875-9a77-bc2a479a6fcb None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:35.107340 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c958e987-9b9f-4875-9a77-bc2a479a6fcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:35.116344 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:34:35.128616 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-c958e987-9b9f-4875-9a77-bc2a479a6fcb None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:34:35.129803 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-c958e987-9b9f-4875-9a77-bc2a479a6fcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:34:35.163101 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987860f5-4096-4b53-b0df-e07b832c2a49 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:35.195481 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 888/1780] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:34:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 31 12:34:35.203788 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990bb6a6-ea76-4a5a-8fe1-9c4f71cfb9c4 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:35.203788 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990bb6a6-ea76-4a5a-8fe1-9c4f71cfb9c4 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:35.205117 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990bb6a6-ea76-4a5a-8fe1-9c4f71cfb9c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:35.221055 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705a7faf-47dc-485d-a27b-869839a136db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:35.221683 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705a7faf-47dc-485d-a27b-869839a136db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:35.221959 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705a7faf-47dc-485d-a27b-869839a136db admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:35.222227 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705a7faf-47dc-485d-a27b-869839a136db admin admin] PATH_INFO: `/v3/users/68cb861cbd0c41ccbc2157980efa87b3` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:35.223434 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705a7faf-47dc-485d-a27b-869839a136db admin admin] RBAC: Authorizing `identity:delete_user(user_id=68cb861cbd0c41ccbc2157980efa87b3)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:35.224622 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-990bb6a6-ea76-4a5a-8fe1-9c4f71cfb9c4 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:34:35.225340 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705a7faf-47dc-485d-a27b-869839a136db admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:35.225809 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-990bb6a6-ea76-4a5a-8fe1-9c4f71cfb9c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:34:35.234145 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 889/1781] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:34:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 31 12:34:35.239077 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990bb6a6-ea76-4a5a-8fe1-9c4f71cfb9c4 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:35.248948 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990bb6a6-ea76-4a5a-8fe1-9c4f71cfb9c4 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:35.253840 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda15555-402f-4a43-9f5b-7ed98fa7fc11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:35.254446 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda15555-402f-4a43-9f5b-7ed98fa7fc11 admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:35.254724 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda15555-402f-4a43-9f5b-7ed98fa7fc11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:35.254992 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda15555-402f-4a43-9f5b-7ed98fa7fc11 admin admin] PATH_INFO: `/v3/domains` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:35.255679 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda15555-402f-4a43-9f5b-7ed98fa7fc11 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:34:35.255999 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda15555-402f-4a43-9f5b-7ed98fa7fc11 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:35.257552 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda15555-402f-4a43-9f5b-7ed98fa7fc11 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:35.270371 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 890/1782] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Oct 31 12:34:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:34:35.274922 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda15555-402f-4a43-9f5b-7ed98fa7fc11 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:35.276835 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-705a7faf-47dc-485d-a27b-869839a136db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68cb861cbd0c41ccbc2157980efa87b3'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:35.285771 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-705a7faf-47dc-485d-a27b-869839a136db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68cb861cbd0c41ccbc2157980efa87b3'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:35.286470 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-705a7faf-47dc-485d-a27b-869839a136db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68cb861cbd0c41ccbc2157980efa87b3'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:35.287237 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-705a7faf-47dc-485d-a27b-869839a136db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68cb861cbd0c41ccbc2157980efa87b3'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:35.288270 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda15555-402f-4a43-9f5b-7ed98fa7fc11 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:35.293234 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09888fea-d0e9-48df-a034-0d2cdb3f3b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:35.293943 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09888fea-d0e9-48df-a034-0d2cdb3f3b0d admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:35.294224 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09888fea-d0e9-48df-a034-0d2cdb3f3b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:35.294537 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09888fea-d0e9-48df-a034-0d2cdb3f3b0d admin admin] PATH_INFO: `/v3/projects` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:35.295264 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09888fea-d0e9-48df-a034-0d2cdb3f3b0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:35.297178 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09888fea-d0e9-48df-a034-0d2cdb3f3b0d admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:35.298247 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:34:35.300510 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-705a7faf-47dc-485d-a27b-869839a136db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68cb861cbd0c41ccbc2157980efa87b3'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:35.317430 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 893/1783] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:34:35 2022] DELETE /identity/v3/users/68cb861cbd0c41ccbc2157980efa87b3 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 31 12:34:35.335246 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 891/1784] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:34:35 2022] POST /identity/v3/projects => generated 359 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 31 12:34:35.336771 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705a7faf-47dc-485d-a27b-869839a136db admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:35.338201 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-705a7faf-47dc-485d-a27b-869839a136db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:34:35.413011 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705a7faf-47dc-485d-a27b-869839a136db admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:35.488157 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019315ed-ed57-4d2e-8fc9-de413008c9cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:35.488791 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019315ed-ed57-4d2e-8fc9-de413008c9cc admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:35.489835 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019315ed-ed57-4d2e-8fc9-de413008c9cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:35.489835 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019315ed-ed57-4d2e-8fc9-de413008c9cc admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:35.490026 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019315ed-ed57-4d2e-8fc9-de413008c9cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:35.493417 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019315ed-ed57-4d2e-8fc9-de413008c9cc admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:35.518708 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 894/1785] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:34:35 2022] POST /identity/v3/users => generated 280 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 31 12:34:35.520039 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09888fea-d0e9-48df-a034-0d2cdb3f3b0d admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:35.530552 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09888fea-d0e9-48df-a034-0d2cdb3f3b0d admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:35.535900 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9111a86-775f-47d9-8966-33c0b7ca3f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:35.536503 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9111a86-775f-47d9-8966-33c0b7ca3f5a admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:35.536829 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9111a86-775f-47d9-8966-33c0b7ca3f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:35.537099 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9111a86-775f-47d9-8966-33c0b7ca3f5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:35.537830 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9111a86-775f-47d9-8966-33c0b7ca3f5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:35.539447 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9111a86-775f-47d9-8966-33c0b7ca3f5a admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:35.552635 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 892/1786] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:34:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:34:35.553869 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019315ed-ed57-4d2e-8fc9-de413008c9cc admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:35.564839 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019315ed-ed57-4d2e-8fc9-de413008c9cc admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:35.569642 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b340e3-3c58-462a-913a-bb36a992f2e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:35.570243 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b340e3-3c58-462a-913a-bb36a992f2e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:35.570520 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b340e3-3c58-462a-913a-bb36a992f2e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:35.570786 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b340e3-3c58-462a-913a-bb36a992f2e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:35.571503 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b340e3-3c58-462a-913a-bb36a992f2e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:35.573089 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b340e3-3c58-462a-913a-bb36a992f2e4 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:35.588252 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 895/1787] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:34:35 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 31 12:34:35.592770 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9111a86-775f-47d9-8966-33c0b7ca3f5a admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:35.606432 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b340e3-3c58-462a-913a-bb36a992f2e4 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:35.606684 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9111a86-775f-47d9-8966-33c0b7ca3f5a admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:35.608400 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-24b340e3-3c58-462a-913a-bb36a992f2e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:34:35.619787 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b88409-76e6-40a5-b2bf-64563c9b3a7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:35.620880 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b88409-76e6-40a5-b2bf-64563c9b3a7e admin admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:35.621392 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b88409-76e6-40a5-b2bf-64563c9b3a7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:35.621670 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b88409-76e6-40a5-b2bf-64563c9b3a7e admin admin] PATH_INFO: `/v3/projects/f07d0f58ce774f03b519526b18a69d37/users/6ab36e845b514205b6a652827dab9d76/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:35.630406 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b340e3-3c58-462a-913a-bb36a992f2e4 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:35.636576 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814766c2-f348-4e2e-931d-22a648f1548a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:35.637236 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814766c2-f348-4e2e-931d-22a648f1548a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:35.637766 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814766c2-f348-4e2e-931d-22a648f1548a admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:35.638050 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814766c2-f348-4e2e-931d-22a648f1548a admin admin] PATH_INFO: `/v3/projects/692870f0e1ac467c809e6597fdb992e1` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:35.639373 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814766c2-f348-4e2e-931d-22a648f1548a admin admin] RBAC: Authorizing `identity:delete_project(project_id=692870f0e1ac467c809e6597fdb992e1)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:35.641380 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814766c2-f348-4e2e-931d-22a648f1548a admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:35.645120 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b88409-76e6-40a5-b2bf-64563c9b3a7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f07d0f58ce774f03b519526b18a69d37, user_id=6ab36e845b514205b6a652827dab9d76, role_id=ca1a5695f8bd499aa0b8cedaeecd2cc6)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:35.648907 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b88409-76e6-40a5-b2bf-64563c9b3a7e admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:35.677432 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 893/1788] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:34:35 2022] PUT /identity/v3/projects/f07d0f58ce774f03b519526b18a69d37/users/6ab36e845b514205b6a652827dab9d76/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:34:35.681698 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef17f70-6f71-4855-b3e3-e7c8609e5e8f None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:35.681978 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef17f70-6f71-4855-b3e3-e7c8609e5e8f None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:35.682248 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef17f70-6f71-4855-b3e3-e7c8609e5e8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:35.732663 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-814766c2-f348-4e2e-931d-22a648f1548a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '692870f0e1ac467c809e6597fdb992e1', 'request_id': 'req-814766c2-f348-4e2e-931d-22a648f1548a'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:35.734189 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-1ef17f70-6f71-4855-b3e3-e7c8609e5e8f None None] MFA Rules not processed for user `6ab36e845b514205b6a652827dab9d76`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:34:35.735751 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-1ef17f70-6f71-4855-b3e3-e7c8609e5e8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:34:35.739712 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-814766c2-f348-4e2e-931d-22a648f1548a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '692870f0e1ac467c809e6597fdb992e1', 'request_id': 'req-814766c2-f348-4e2e-931d-22a648f1548a'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:35.749004 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-814766c2-f348-4e2e-931d-22a648f1548a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '692870f0e1ac467c809e6597fdb992e1', 'request_id': 'req-814766c2-f348-4e2e-931d-22a648f1548a'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:35.754944 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:34:35.772509 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-814766c2-f348-4e2e-931d-22a648f1548a admin admin] The token cache is being invalidate because project 692870f0e1ac467c809e6597fdb992e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:34:35.772509 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-814766c2-f348-4e2e-931d-22a648f1548a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:35.773062 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-814766c2-f348-4e2e-931d-22a648f1548a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:35.775122 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 896/1789] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:34:35 2022] DELETE /identity/v3/projects/692870f0e1ac467c809e6597fdb992e1 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:34:35.780084 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eb948d-b26d-4aab-8346-1c240d7a4c3e None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:35.780361 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eb948d-b26d-4aab-8346-1c240d7a4c3e None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:35.780626 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eb948d-b26d-4aab-8346-1c240d7a4c3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:35.805151 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-76eb948d-b26d-4aab-8346-1c240d7a4c3e None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:34:35.806653 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-76eb948d-b26d-4aab-8346-1c240d7a4c3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:34:35.822663 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 894/1790] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:34:35 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 31 12:34:35.835050 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef17f70-6f71-4855-b3e3-e7c8609e5e8f None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:35.837113 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-1ef17f70-6f71-4855-b3e3-e7c8609e5e8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:34:35.860424 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:34:35.872515 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 897/1791] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:34:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:34:35.880264 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba23ad57-59fd-4a55-8f35-6aa4aa387cfd None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:35.880548 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba23ad57-59fd-4a55-8f35-6aa4aa387cfd None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:35.880806 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba23ad57-59fd-4a55-8f35-6aa4aa387cfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:35.901063 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-ba23ad57-59fd-4a55-8f35-6aa4aa387cfd None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:34:35.902396 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-ba23ad57-59fd-4a55-8f35-6aa4aa387cfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:34:35.908927 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef17f70-6f71-4855-b3e3-e7c8609e5e8f None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:35.912301 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75ca0a4-9a82-4c6a-9da6-4ee43ee322f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d6d7ea077d49e8a66c6c6762499702', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:35.912737 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 898/1792] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:34:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 31 12:34:35.912914 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75ca0a4-9a82-4c6a-9da6-4ee43ee322f0 service cinder] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:35.913195 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75ca0a4-9a82-4c6a-9da6-4ee43ee322f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:35.913553 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75ca0a4-9a82-4c6a-9da6-4ee43ee322f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:35.916855 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba23ad57-59fd-4a55-8f35-6aa4aa387cfd None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:35.924873 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75ca0a4-9a82-4c6a-9da6-4ee43ee322f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:35.926312 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75ca0a4-9a82-4c6a-9da6-4ee43ee322f0 service cinder] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:35.926550 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba23ad57-59fd-4a55-8f35-6aa4aa387cfd None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:35.931453 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6250c7d3-b6ff-4255-b7a4-f1691c17ed7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:35.931972 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6250c7d3-b6ff-4255-b7a4-f1691c17ed7a admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:35.932316 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6250c7d3-b6ff-4255-b7a4-f1691c17ed7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:35.932655 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6250c7d3-b6ff-4255-b7a4-f1691c17ed7a admin admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:35.933364 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6250c7d3-b6ff-4255-b7a4-f1691c17ed7a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:34:35.933732 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6250c7d3-b6ff-4255-b7a4-f1691c17ed7a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:35.935400 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6250c7d3-b6ff-4255-b7a4-f1691c17ed7a admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:35.936847 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 895/1793] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:34:35 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 31 12:34:35.960050 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 899/1794] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Oct 31 12:34:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 31 12:34:35.964263 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75ca0a4-9a82-4c6a-9da6-4ee43ee322f0 service cinder] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:35.978466 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75ca0a4-9a82-4c6a-9da6-4ee43ee322f0 service cinder] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:35.984645 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622535e0-f7a9-4711-952b-f3b0cf6e630c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:35.985599 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622535e0-f7a9-4711-952b-f3b0cf6e630c admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:35.985886 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622535e0-f7a9-4711-952b-f3b0cf6e630c admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:35.986161 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622535e0-f7a9-4711-952b-f3b0cf6e630c admin admin] PATH_INFO: `/v3/projects` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:35.987044 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622535e0-f7a9-4711-952b-f3b0cf6e630c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:35.989179 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622535e0-f7a9-4711-952b-f3b0cf6e630c admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:36.043840 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 896/1795] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:34:35 2022] POST /identity/v3/projects => generated 377 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 31 12:34:36.048818 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6250c7d3-b6ff-4255-b7a4-f1691c17ed7a admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:36.063364 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6250c7d3-b6ff-4255-b7a4-f1691c17ed7a admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:36.142114 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ebc09f-27c8-4c9b-9cbe-d7807465f8c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:36.142850 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ebc09f-27c8-4c9b-9cbe-d7807465f8c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:36.142962 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ebc09f-27c8-4c9b-9cbe-d7807465f8c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:36.143414 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ebc09f-27c8-4c9b-9cbe-d7807465f8c3 admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:36.144121 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ebc09f-27c8-4c9b-9cbe-d7807465f8c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:36.146386 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ebc09f-27c8-4c9b-9cbe-d7807465f8c3 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:36.177669 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 900/1796] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:34:36 2022] POST /identity/v3/users => generated 289 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:34:36.180006 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622535e0-f7a9-4711-952b-f3b0cf6e630c admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:36.192592 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622535e0-f7a9-4711-952b-f3b0cf6e630c admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:36.202255 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d151c066-453e-45f5-ab09-7a5693011b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:36.203157 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d151c066-453e-45f5-ab09-7a5693011b1c admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:36.203570 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d151c066-453e-45f5-ab09-7a5693011b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:36.203994 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d151c066-453e-45f5-ab09-7a5693011b1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:36.204807 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d151c066-453e-45f5-ab09-7a5693011b1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:36.206637 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d151c066-453e-45f5-ab09-7a5693011b1c admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:36.224676 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 897/1797] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:34:36 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:34:36.229402 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ebc09f-27c8-4c9b-9cbe-d7807465f8c3 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:36.243613 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ebc09f-27c8-4c9b-9cbe-d7807465f8c3 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:36.250043 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd82747-d983-4d9a-b2cd-83c7f6bdb3d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:36.250518 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd82747-d983-4d9a-b2cd-83c7f6bdb3d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:36.250835 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd82747-d983-4d9a-b2cd-83c7f6bdb3d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:36.251835 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd82747-d983-4d9a-b2cd-83c7f6bdb3d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:36.252980 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd82747-d983-4d9a-b2cd-83c7f6bdb3d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:36.261117 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd82747-d983-4d9a-b2cd-83c7f6bdb3d1 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:36.290023 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 901/1798] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:34:36 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:34:36.294252 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d151c066-453e-45f5-ab09-7a5693011b1c admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:36.305991 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d151c066-453e-45f5-ab09-7a5693011b1c admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:36.311199 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c542f1d7-611f-4dd2-a82d-cd7c5584b023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:36.311947 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c542f1d7-611f-4dd2-a82d-cd7c5584b023 admin admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:36.312237 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c542f1d7-611f-4dd2-a82d-cd7c5584b023 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:36.312515 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c542f1d7-611f-4dd2-a82d-cd7c5584b023 admin admin] PATH_INFO: `/v3/projects/bd3f95a73e734b8199e8843ce87c59b9/users/a0e823bcb75b43fa94a3aa75e3fc6152/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:36.333225 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c542f1d7-611f-4dd2-a82d-cd7c5584b023 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd3f95a73e734b8199e8843ce87c59b9, user_id=a0e823bcb75b43fa94a3aa75e3fc6152, role_id=ca1a5695f8bd499aa0b8cedaeecd2cc6)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:36.340593 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c542f1d7-611f-4dd2-a82d-cd7c5584b023 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:36.362933 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 898/1799] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:34:36 2022] PUT /identity/v3/projects/bd3f95a73e734b8199e8843ce87c59b9/users/a0e823bcb75b43fa94a3aa75e3fc6152/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:34:36.373360 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6be639a-ba12-4aa2-b059-d841a7dfeece None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:36.373826 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6be639a-ba12-4aa2-b059-d841a7dfeece None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:36.374257 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6be639a-ba12-4aa2-b059-d841a7dfeece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:36.416392 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-f6be639a-ba12-4aa2-b059-d841a7dfeece None None] MFA Rules not processed for user `a0e823bcb75b43fa94a3aa75e3fc6152`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:34:36.417686 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-f6be639a-ba12-4aa2-b059-d841a7dfeece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:34:36.460487 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:34:36.487787 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 902/1800] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:34:36 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:34:36.492295 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c542f1d7-611f-4dd2-a82d-cd7c5584b023 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:36.555630 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c542f1d7-611f-4dd2-a82d-cd7c5584b023 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:36.560478 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd12875-25f0-4778-83a4-881e86063afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:36.561071 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd12875-25f0-4778-83a4-881e86063afe admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:36.561350 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd12875-25f0-4778-83a4-881e86063afe admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:36.561617 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd12875-25f0-4778-83a4-881e86063afe admin admin] PATH_INFO: `/v3/projects` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:36.562228 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd12875-25f0-4778-83a4-881e86063afe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:36.564374 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd12875-25f0-4778-83a4-881e86063afe admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:36.591703 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 899/1801] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:34:36 2022] POST /identity/v3/projects => generated 377 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:34:36.597002 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6be639a-ba12-4aa2-b059-d841a7dfeece None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:36.609757 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6be639a-ba12-4aa2-b059-d841a7dfeece None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:36.666139 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f459bc13-e4ae-4344-97b9-dc1f4f5eaa41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:36.666791 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f459bc13-e4ae-4344-97b9-dc1f4f5eaa41 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:36.667072 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f459bc13-e4ae-4344-97b9-dc1f4f5eaa41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:36.667373 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f459bc13-e4ae-4344-97b9-dc1f4f5eaa41 admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:36.667998 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f459bc13-e4ae-4344-97b9-dc1f4f5eaa41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:36.669943 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f459bc13-e4ae-4344-97b9-dc1f4f5eaa41 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:36.692221 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 903/1802] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:34:36 2022] POST /identity/v3/users => generated 289 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:34:36.696542 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd12875-25f0-4778-83a4-881e86063afe admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:36.707384 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd12875-25f0-4778-83a4-881e86063afe admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:36.711969 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2041634-d951-4d5b-beb1-9c2cd5550914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:36.712509 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2041634-d951-4d5b-beb1-9c2cd5550914 admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:36.712783 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2041634-d951-4d5b-beb1-9c2cd5550914 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:36.713056 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2041634-d951-4d5b-beb1-9c2cd5550914 admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:36.713679 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2041634-d951-4d5b-beb1-9c2cd5550914 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:36.715281 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2041634-d951-4d5b-beb1-9c2cd5550914 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:36.730266 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 900/1803] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:34:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 31 12:34:36.731966 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f459bc13-e4ae-4344-97b9-dc1f4f5eaa41 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:36.742572 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f459bc13-e4ae-4344-97b9-dc1f4f5eaa41 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:36.749978 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5653412-2ba0-41da-97ad-c9d45cd12fd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:36.750635 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5653412-2ba0-41da-97ad-c9d45cd12fd4 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:36.750866 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5653412-2ba0-41da-97ad-c9d45cd12fd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:36.751132 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5653412-2ba0-41da-97ad-c9d45cd12fd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:36.751977 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5653412-2ba0-41da-97ad-c9d45cd12fd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:36.753690 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5653412-2ba0-41da-97ad-c9d45cd12fd4 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:36.765004 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 904/1804] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:34:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:34:36.770247 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2041634-d951-4d5b-beb1-9c2cd5550914 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:36.780606 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2041634-d951-4d5b-beb1-9c2cd5550914 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:36.786088 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb91a92-5767-4b48-b878-cea7cce20601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:36.786739 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb91a92-5767-4b48-b878-cea7cce20601 admin admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:36.787016 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb91a92-5767-4b48-b878-cea7cce20601 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:36.787321 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb91a92-5767-4b48-b878-cea7cce20601 admin admin] PATH_INFO: `/v3/projects/8bf248f7932a47e188fdbc53c9800788/users/76c026574ba140e38d0012284db1ae96/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:36.801955 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb91a92-5767-4b48-b878-cea7cce20601 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bf248f7932a47e188fdbc53c9800788, user_id=76c026574ba140e38d0012284db1ae96, role_id=ca1a5695f8bd499aa0b8cedaeecd2cc6)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:36.804600 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb91a92-5767-4b48-b878-cea7cce20601 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:36.817406 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 901/1805] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:34:36 2022] PUT /identity/v3/projects/8bf248f7932a47e188fdbc53c9800788/users/76c026574ba140e38d0012284db1ae96/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:34:36.830455 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc764063-cae5-4b86-a384-e47cb30490f7 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:36.830767 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc764063-cae5-4b86-a384-e47cb30490f7 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:36.831056 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc764063-cae5-4b86-a384-e47cb30490f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:36.865931 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-bc764063-cae5-4b86-a384-e47cb30490f7 None None] MFA Rules not processed for user `76c026574ba140e38d0012284db1ae96`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:34:36.867151 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-bc764063-cae5-4b86-a384-e47cb30490f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:34:36.942383 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 905/1806] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:34:36 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:34:36.948549 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb91a92-5767-4b48-b878-cea7cce20601 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:36.963024 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:34:37.019648 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb91a92-5767-4b48-b878-cea7cce20601 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:37.024634 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9dc23b-262a-4e90-ab66-39d7ebc7084e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:37.025239 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9dc23b-262a-4e90-ab66-39d7ebc7084e admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:37.025516 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9dc23b-262a-4e90-ab66-39d7ebc7084e admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:37.025785 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9dc23b-262a-4e90-ab66-39d7ebc7084e admin admin] PATH_INFO: `/v3/projects` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:37.026427 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9dc23b-262a-4e90-ab66-39d7ebc7084e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:37.028360 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9dc23b-262a-4e90-ab66-39d7ebc7084e admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:37.057234 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 902/1807] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:34:36 2022] POST /identity/v3/projects => generated 377 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:34:37.060654 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc764063-cae5-4b86-a384-e47cb30490f7 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:37.071123 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc764063-cae5-4b86-a384-e47cb30490f7 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:37.119458 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:34:37.131552 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbb296c-f791-4cd5-8b2c-18ea419f2ad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:37.132173 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbb296c-f791-4cd5-8b2c-18ea419f2ad0 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:37.132454 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbb296c-f791-4cd5-8b2c-18ea419f2ad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:37.132734 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbb296c-f791-4cd5-8b2c-18ea419f2ad0 admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:37.133373 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbb296c-f791-4cd5-8b2c-18ea419f2ad0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:37.135916 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbb296c-f791-4cd5-8b2c-18ea419f2ad0 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:37.159265 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 906/1808] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:34:37 2022] POST /identity/v3/users => generated 295 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:34:37.163315 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9dc23b-262a-4e90-ab66-39d7ebc7084e admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:37.174642 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9dc23b-262a-4e90-ab66-39d7ebc7084e admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:37.179424 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228025bc-4559-4681-a2dd-0efb98b1207f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:37.180124 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228025bc-4559-4681-a2dd-0efb98b1207f admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:37.180409 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228025bc-4559-4681-a2dd-0efb98b1207f admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:37.180681 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228025bc-4559-4681-a2dd-0efb98b1207f admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:37.181324 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228025bc-4559-4681-a2dd-0efb98b1207f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:37.182914 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228025bc-4559-4681-a2dd-0efb98b1207f admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:37.195426 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 903/1809] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:34:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:34:37.200066 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbb296c-f791-4cd5-8b2c-18ea419f2ad0 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:37.210889 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbb296c-f791-4cd5-8b2c-18ea419f2ad0 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:37.215686 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4693741-4715-4b0b-a3dd-125a17c30614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:37.216269 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4693741-4715-4b0b-a3dd-125a17c30614 admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:37.216659 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4693741-4715-4b0b-a3dd-125a17c30614 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:37.216938 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4693741-4715-4b0b-a3dd-125a17c30614 admin admin] PATH_INFO: `/v3/projects/d195666855ad4aed91077289a69087df/users/e1443a31ffbe49c198a41359183705c3/roles/176c07b4fe3b4537bdfeb4be21b379c5` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:37.232219 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4693741-4715-4b0b-a3dd-125a17c30614 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d195666855ad4aed91077289a69087df, user_id=e1443a31ffbe49c198a41359183705c3, role_id=176c07b4fe3b4537bdfeb4be21b379c5)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:37.234858 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4693741-4715-4b0b-a3dd-125a17c30614 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:37.249569 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 907/1810] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:34:37 2022] PUT /identity/v3/projects/d195666855ad4aed91077289a69087df/users/e1443a31ffbe49c198a41359183705c3/roles/176c07b4fe3b4537bdfeb4be21b379c5 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:34:37.253661 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228025bc-4559-4681-a2dd-0efb98b1207f admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:37.263645 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228025bc-4559-4681-a2dd-0efb98b1207f admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:37.323342 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d767e4b-69f8-4423-946a-a35f35115d63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:37.323945 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d767e4b-69f8-4423-946a-a35f35115d63 admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:37.324222 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d767e4b-69f8-4423-946a-a35f35115d63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:37.324495 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d767e4b-69f8-4423-946a-a35f35115d63 admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:37.325121 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d767e4b-69f8-4423-946a-a35f35115d63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:37.326710 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d767e4b-69f8-4423-946a-a35f35115d63 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:37.336925 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 904/1811] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:34:37 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:34:37.341294 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4693741-4715-4b0b-a3dd-125a17c30614 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:37.352874 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4693741-4715-4b0b-a3dd-125a17c30614 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:37.359023 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e81e3d-e9fc-4d15-9dd8-78f9901aed8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:37.359736 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e81e3d-e9fc-4d15-9dd8-78f9901aed8a admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:37.360115 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e81e3d-e9fc-4d15-9dd8-78f9901aed8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:37.360299 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e81e3d-e9fc-4d15-9dd8-78f9901aed8a admin admin] PATH_INFO: `/v3/projects/d195666855ad4aed91077289a69087df/users/e1443a31ffbe49c198a41359183705c3/roles/176c07b4fe3b4537bdfeb4be21b379c5` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:37.362522 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e81e3d-e9fc-4d15-9dd8-78f9901aed8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d195666855ad4aed91077289a69087df, user_id=e1443a31ffbe49c198a41359183705c3, role_id=176c07b4fe3b4537bdfeb4be21b379c5)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:37.365942 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e81e3d-e9fc-4d15-9dd8-78f9901aed8a admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:37.377996 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 908/1812] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:34:37 2022] PUT /identity/v3/projects/d195666855ad4aed91077289a69087df/users/e1443a31ffbe49c198a41359183705c3/roles/176c07b4fe3b4537bdfeb4be21b379c5 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:34:37.388356 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a765e6-567f-4e4f-ac94-0ee2e8c91339 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:37.388685 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a765e6-567f-4e4f-ac94-0ee2e8c91339 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:37.388965 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a765e6-567f-4e4f-ac94-0ee2e8c91339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:37.425204 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-a4a765e6-567f-4e4f-ac94-0ee2e8c91339 None None] MFA Rules not processed for user `e1443a31ffbe49c198a41359183705c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:34:37.426425 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-a4a765e6-567f-4e4f-ac94-0ee2e8c91339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:34:37.497785 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 905/1813] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:34:37 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:34:37.514380 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e81e3d-e9fc-4d15-9dd8-78f9901aed8a admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:37.569517 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e81e3d-e9fc-4d15-9dd8-78f9901aed8a admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:37.573226 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aba53c1-39df-4e24-a63d-f60099221492 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d6d7ea077d49e8a66c6c6762499702', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:37.573833 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aba53c1-39df-4e24-a63d-f60099221492 service cinder] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:37.574113 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aba53c1-39df-4e24-a63d-f60099221492 service cinder] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:37.574381 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aba53c1-39df-4e24-a63d-f60099221492 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:37.637657 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aba53c1-39df-4e24-a63d-f60099221492 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:37.639135 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aba53c1-39df-4e24-a63d-f60099221492 service cinder] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:37.649292 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 909/1814] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:34:37 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:34:39.200943 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a765e6-567f-4e4f-ac94-0ee2e8c91339 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:39.211073 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a765e6-567f-4e4f-ac94-0ee2e8c91339 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:39.214789 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07763b68-3faa-42fc-9101-0bf949806c94 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d6d7ea077d49e8a66c6c6762499702', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:39.215466 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07763b68-3faa-42fc-9101-0bf949806c94 service cinder] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:39.215742 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07763b68-3faa-42fc-9101-0bf949806c94 service cinder] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:39.216018 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07763b68-3faa-42fc-9101-0bf949806c94 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:39.222042 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:34:39.278430 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07763b68-3faa-42fc-9101-0bf949806c94 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:39.279906 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07763b68-3faa-42fc-9101-0bf949806c94 service cinder] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:39.289612 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 906/1815] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:34:39 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:34:39.851061 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aba53c1-39df-4e24-a63d-f60099221492 service cinder] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:39.859550 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aba53c1-39df-4e24-a63d-f60099221492 service cinder] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:39.864351 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27838d4-cc04-4deb-b167-3b9256096187 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d6d7ea077d49e8a66c6c6762499702', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:39.864955 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27838d4-cc04-4deb-b167-3b9256096187 service cinder] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:39.865230 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27838d4-cc04-4deb-b167-3b9256096187 service cinder] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:39.865501 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27838d4-cc04-4deb-b167-3b9256096187 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:39.874561 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27838d4-cc04-4deb-b167-3b9256096187 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:39.876074 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27838d4-cc04-4deb-b167-3b9256096187 service cinder] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:39.885487 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 910/1816] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:34:39 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:34:43.753381 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07763b68-3faa-42fc-9101-0bf949806c94 service cinder] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:43.818318 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07763b68-3faa-42fc-9101-0bf949806c94 service cinder] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:43.822986 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efb34c7-f39b-4065-b60d-5e38814ed434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:43.823717 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efb34c7-f39b-4065-b60d-5e38814ed434 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:43.824011 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efb34c7-f39b-4065-b60d-5e38814ed434 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:43.824267 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efb34c7-f39b-4065-b60d-5e38814ed434 admin admin] PATH_INFO: `/v3/users/a0e823bcb75b43fa94a3aa75e3fc6152` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:43.825377 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efb34c7-f39b-4065-b60d-5e38814ed434 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0e823bcb75b43fa94a3aa75e3fc6152)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:43.827582 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efb34c7-f39b-4065-b60d-5e38814ed434 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:43.879561 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-8efb34c7-f39b-4065-b60d-5e38814ed434 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0e823bcb75b43fa94a3aa75e3fc6152'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:43.887155 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-8efb34c7-f39b-4065-b60d-5e38814ed434 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0e823bcb75b43fa94a3aa75e3fc6152'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:43.887795 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-8efb34c7-f39b-4065-b60d-5e38814ed434 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0e823bcb75b43fa94a3aa75e3fc6152'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:43.902235 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-8efb34c7-f39b-4065-b60d-5e38814ed434 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0e823bcb75b43fa94a3aa75e3fc6152'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:43.910803 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-8efb34c7-f39b-4065-b60d-5e38814ed434 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0e823bcb75b43fa94a3aa75e3fc6152'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:43.916793 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 907/1817] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:34:43 2022] DELETE /identity/v3/users/a0e823bcb75b43fa94a3aa75e3fc6152 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:34:43.921237 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50492893-7571-4af6-b0b2-b9edb8f24d25 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:43.921698 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50492893-7571-4af6-b0b2-b9edb8f24d25 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:43.921838 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50492893-7571-4af6-b0b2-b9edb8f24d25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:43.947732 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-50492893-7571-4af6-b0b2-b9edb8f24d25 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:34:43.948929 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-50492893-7571-4af6-b0b2-b9edb8f24d25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:34:44.006060 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:34:44.031061 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 911/1818] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:34:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:34:44.043060 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efb34c7-f39b-4065-b60d-5e38814ed434 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:44.044578 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-8efb34c7-f39b-4065-b60d-5e38814ed434 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:34:44.136275 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:34:44.176545 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efb34c7-f39b-4065-b60d-5e38814ed434 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:44.180968 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2470af-b492-4800-baa9-2f6c6eb7e65d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:44.181579 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2470af-b492-4800-baa9-2f6c6eb7e65d service neutron] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:44.181863 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2470af-b492-4800-baa9-2f6c6eb7e65d service neutron] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:44.182134 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2470af-b492-4800-baa9-2f6c6eb7e65d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:44.192968 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2470af-b492-4800-baa9-2f6c6eb7e65d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:44.194437 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2470af-b492-4800-baa9-2f6c6eb7e65d service neutron] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:44.208056 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 908/1819] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:34:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:34:44.513757 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50492893-7571-4af6-b0b2-b9edb8f24d25 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:44.515522 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-50492893-7571-4af6-b0b2-b9edb8f24d25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:34:44.531118 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50492893-7571-4af6-b0b2-b9edb8f24d25 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:44.536926 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cd411b-2cb1-4ee0-ba0d-db7e9852bec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:44.537575 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cd411b-2cb1-4ee0-ba0d-db7e9852bec7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:44.537855 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cd411b-2cb1-4ee0-ba0d-db7e9852bec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:44.538126 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cd411b-2cb1-4ee0-ba0d-db7e9852bec7 admin admin] PATH_INFO: `/v3/projects/bd3f95a73e734b8199e8843ce87c59b9` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:44.539445 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cd411b-2cb1-4ee0-ba0d-db7e9852bec7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd3f95a73e734b8199e8843ce87c59b9)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:44.541499 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cd411b-2cb1-4ee0-ba0d-db7e9852bec7 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:44.593626 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-f1cd411b-2cb1-4ee0-ba0d-db7e9852bec7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd3f95a73e734b8199e8843ce87c59b9', 'request_id': 'req-f1cd411b-2cb1-4ee0-ba0d-db7e9852bec7'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:44.600409 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-f1cd411b-2cb1-4ee0-ba0d-db7e9852bec7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd3f95a73e734b8199e8843ce87c59b9', 'request_id': 'req-f1cd411b-2cb1-4ee0-ba0d-db7e9852bec7'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:44.608078 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-f1cd411b-2cb1-4ee0-ba0d-db7e9852bec7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd3f95a73e734b8199e8843ce87c59b9', 'request_id': 'req-f1cd411b-2cb1-4ee0-ba0d-db7e9852bec7'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:44.619785 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-f1cd411b-2cb1-4ee0-ba0d-db7e9852bec7 admin admin] The token cache is being invalidate because project bd3f95a73e734b8199e8843ce87c59b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:34:44.620081 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-f1cd411b-2cb1-4ee0-ba0d-db7e9852bec7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:44.620728 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-f1cd411b-2cb1-4ee0-ba0d-db7e9852bec7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:44.622218 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 912/1820] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:34:44 2022] DELETE /identity/v3/projects/bd3f95a73e734b8199e8843ce87c59b9 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:34:44.626954 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2470af-b492-4800-baa9-2f6c6eb7e65d service neutron] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:44.637126 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2470af-b492-4800-baa9-2f6c6eb7e65d service neutron] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:44.709921 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988592c0-391b-42c6-9c03-918691a06654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:44.710560 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988592c0-391b-42c6-9c03-918691a06654 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:44.710837 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988592c0-391b-42c6-9c03-918691a06654 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:44.711428 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988592c0-391b-42c6-9c03-918691a06654 admin admin] PATH_INFO: `/v3/users/76c026574ba140e38d0012284db1ae96` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:44.712539 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988592c0-391b-42c6-9c03-918691a06654 admin admin] RBAC: Authorizing `identity:delete_user(user_id=76c026574ba140e38d0012284db1ae96)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:44.714480 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988592c0-391b-42c6-9c03-918691a06654 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:44.767960 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-988592c0-391b-42c6-9c03-918691a06654 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76c026574ba140e38d0012284db1ae96'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:44.776565 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-988592c0-391b-42c6-9c03-918691a06654 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76c026574ba140e38d0012284db1ae96'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:44.777242 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-988592c0-391b-42c6-9c03-918691a06654 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76c026574ba140e38d0012284db1ae96'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:44.790324 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-988592c0-391b-42c6-9c03-918691a06654 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76c026574ba140e38d0012284db1ae96'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:44.798899 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-988592c0-391b-42c6-9c03-918691a06654 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76c026574ba140e38d0012284db1ae96'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:44.805632 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 909/1821] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:34:44 2022] DELETE /identity/v3/users/76c026574ba140e38d0012284db1ae96 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:34:45.080335 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cd411b-2cb1-4ee0-ba0d-db7e9852bec7 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:45.082259 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-f1cd411b-2cb1-4ee0-ba0d-db7e9852bec7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:34:45.097286 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cd411b-2cb1-4ee0-ba0d-db7e9852bec7 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:45.158748 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7acf56-a470-4deb-becb-3f7300283bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:45.159429 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7acf56-a470-4deb-becb-3f7300283bc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:45.159711 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7acf56-a470-4deb-becb-3f7300283bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:45.159988 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7acf56-a470-4deb-becb-3f7300283bc1 admin admin] PATH_INFO: `/v3/projects/8bf248f7932a47e188fdbc53c9800788` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:45.161162 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7acf56-a470-4deb-becb-3f7300283bc1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8bf248f7932a47e188fdbc53c9800788)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:45.163090 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7acf56-a470-4deb-becb-3f7300283bc1 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:45.221927 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-df7acf56-a470-4deb-becb-3f7300283bc1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8bf248f7932a47e188fdbc53c9800788', 'request_id': 'req-df7acf56-a470-4deb-becb-3f7300283bc1'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:45.229584 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-df7acf56-a470-4deb-becb-3f7300283bc1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8bf248f7932a47e188fdbc53c9800788', 'request_id': 'req-df7acf56-a470-4deb-becb-3f7300283bc1'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:45.241843 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-df7acf56-a470-4deb-becb-3f7300283bc1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8bf248f7932a47e188fdbc53c9800788', 'request_id': 'req-df7acf56-a470-4deb-becb-3f7300283bc1'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:45.249360 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:34:45.256272 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-df7acf56-a470-4deb-becb-3f7300283bc1 admin admin] The token cache is being invalidate because project 8bf248f7932a47e188fdbc53c9800788 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:34:45.256743 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-df7acf56-a470-4deb-becb-3f7300283bc1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:45.257852 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-df7acf56-a470-4deb-becb-3f7300283bc1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:45.259606 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 913/1822] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:34:45 2022] DELETE /identity/v3/projects/8bf248f7932a47e188fdbc53c9800788 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:34:45.264245 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988592c0-391b-42c6-9c03-918691a06654 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:45.265685 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-988592c0-391b-42c6-9c03-918691a06654 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:34:45.280075 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988592c0-391b-42c6-9c03-918691a06654 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:45.326312 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:34:45.340106 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbb3cd9-2eaf-4adc-8519-45f5736b09ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:45.340735 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbb3cd9-2eaf-4adc-8519-45f5736b09ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:45.341010 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbb3cd9-2eaf-4adc-8519-45f5736b09ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:45.341307 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbb3cd9-2eaf-4adc-8519-45f5736b09ab admin admin] PATH_INFO: `/v3/users/e1443a31ffbe49c198a41359183705c3` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:45.342409 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbb3cd9-2eaf-4adc-8519-45f5736b09ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1443a31ffbe49c198a41359183705c3)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:45.344392 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbb3cd9-2eaf-4adc-8519-45f5736b09ab admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:45.389587 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-6cbb3cd9-2eaf-4adc-8519-45f5736b09ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1443a31ffbe49c198a41359183705c3'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:45.398372 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-6cbb3cd9-2eaf-4adc-8519-45f5736b09ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1443a31ffbe49c198a41359183705c3'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:45.398985 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-6cbb3cd9-2eaf-4adc-8519-45f5736b09ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1443a31ffbe49c198a41359183705c3'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:45.410951 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-6cbb3cd9-2eaf-4adc-8519-45f5736b09ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1443a31ffbe49c198a41359183705c3'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:45.417727 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-6cbb3cd9-2eaf-4adc-8519-45f5736b09ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1443a31ffbe49c198a41359183705c3'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:45.423576 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 910/1823] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:34:45 2022] DELETE /identity/v3/users/e1443a31ffbe49c198a41359183705c3 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:34:45.699030 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7acf56-a470-4deb-becb-3f7300283bc1 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:45.700564 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-df7acf56-a470-4deb-becb-3f7300283bc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:34:45.714382 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7acf56-a470-4deb-becb-3f7300283bc1 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:45.777421 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c0981d-4947-4e40-a40b-6eab0252b669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:45.778067 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c0981d-4947-4e40-a40b-6eab0252b669 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:45.778346 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c0981d-4947-4e40-a40b-6eab0252b669 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:45.778616 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c0981d-4947-4e40-a40b-6eab0252b669 admin admin] PATH_INFO: `/v3/projects/d195666855ad4aed91077289a69087df` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:45.779908 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c0981d-4947-4e40-a40b-6eab0252b669 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d195666855ad4aed91077289a69087df)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:45.782384 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c0981d-4947-4e40-a40b-6eab0252b669 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:45.829332 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-02c0981d-4947-4e40-a40b-6eab0252b669 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd195666855ad4aed91077289a69087df', 'request_id': 'req-02c0981d-4947-4e40-a40b-6eab0252b669'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:45.835895 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-02c0981d-4947-4e40-a40b-6eab0252b669 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd195666855ad4aed91077289a69087df', 'request_id': 'req-02c0981d-4947-4e40-a40b-6eab0252b669'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:45.843413 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-02c0981d-4947-4e40-a40b-6eab0252b669 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd195666855ad4aed91077289a69087df', 'request_id': 'req-02c0981d-4947-4e40-a40b-6eab0252b669'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:45.854574 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-02c0981d-4947-4e40-a40b-6eab0252b669 admin admin] The token cache is being invalidate because project d195666855ad4aed91077289a69087df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:34:45.854861 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-02c0981d-4947-4e40-a40b-6eab0252b669 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:45.855493 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-02c0981d-4947-4e40-a40b-6eab0252b669 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:45.856997 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 914/1824] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:34:45 2022] DELETE /identity/v3/projects/d195666855ad4aed91077289a69087df => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:34:45.861999 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0d123c-9aba-42a5-9d72-ecf9dbb3943d None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:45.862450 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0d123c-9aba-42a5-9d72-ecf9dbb3943d None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:45.862538 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0d123c-9aba-42a5-9d72-ecf9dbb3943d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:45.882114 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-7c0d123c-9aba-42a5-9d72-ecf9dbb3943d None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:34:45.883349 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-7c0d123c-9aba-42a5-9d72-ecf9dbb3943d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:34:45.942035 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 911/1825] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:34:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:34:45.950029 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d19a5d-3151-4374-b0ab-c529a35cf726 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:45.950315 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d19a5d-3151-4374-b0ab-c529a35cf726 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:45.950591 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d19a5d-3151-4374-b0ab-c529a35cf726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:45.971030 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-76d19a5d-3151-4374-b0ab-c529a35cf726 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:34:45.972254 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-76d19a5d-3151-4374-b0ab-c529a35cf726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:34:45.980097 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 915/1826] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:34:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:34:45.984241 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0d123c-9aba-42a5-9d72-ecf9dbb3943d None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:45.993816 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0d123c-9aba-42a5-9d72-ecf9dbb3943d None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:45.998412 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427b3aea-e63c-45a6-8f53-52fe829bcbd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:45.999003 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427b3aea-e63c-45a6-8f53-52fe829bcbd5 admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:45.999313 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427b3aea-e63c-45a6-8f53-52fe829bcbd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:45.999582 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427b3aea-e63c-45a6-8f53-52fe829bcbd5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:46.000211 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427b3aea-e63c-45a6-8f53-52fe829bcbd5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:34:46.001181 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427b3aea-e63c-45a6-8f53-52fe829bcbd5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:46.002759 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427b3aea-e63c-45a6-8f53-52fe829bcbd5 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:46.014223 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 912/1827] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Oct 31 12:34:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:34:46.018983 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d19a5d-3151-4374-b0ab-c529a35cf726 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:46.029634 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d19a5d-3151-4374-b0ab-c529a35cf726 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:46.035733 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c06262b-e369-4949-8002-f4c33f9ad8ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:46.036344 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c06262b-e369-4949-8002-f4c33f9ad8ff admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:46.036624 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c06262b-e369-4949-8002-f4c33f9ad8ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:46.036898 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c06262b-e369-4949-8002-f4c33f9ad8ff admin admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:46.037538 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c06262b-e369-4949-8002-f4c33f9ad8ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:46.039534 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c06262b-e369-4949-8002-f4c33f9ad8ff admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:46.067551 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 916/1828] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:34:46 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:34:46.072095 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427b3aea-e63c-45a6-8f53-52fe829bcbd5 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:46.082296 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427b3aea-e63c-45a6-8f53-52fe829bcbd5 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:46.133855 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:34:46.142077 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9631d7c-d7da-44a5-a158-884edee8b2c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:46.142842 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9631d7c-d7da-44a5-a158-884edee8b2c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:46.143351 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9631d7c-d7da-44a5-a158-884edee8b2c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:46.143671 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9631d7c-d7da-44a5-a158-884edee8b2c4 admin admin] PATH_INFO: `/v3/users` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:46.144488 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9631d7c-d7da-44a5-a158-884edee8b2c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:46.146801 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9631d7c-d7da-44a5-a158-884edee8b2c4 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:46.173487 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 913/1829] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:34:46 2022] POST /identity/v3/users => generated 293 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:34:46.175659 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c06262b-e369-4949-8002-f4c33f9ad8ff admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:46.187567 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c06262b-e369-4949-8002-f4c33f9ad8ff admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:46.192562 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f21b9ca-f22d-42c7-aaeb-d653ac318ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:46.193173 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f21b9ca-f22d-42c7-aaeb-d653ac318ef7 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:46.193447 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f21b9ca-f22d-42c7-aaeb-d653ac318ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:46.193714 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f21b9ca-f22d-42c7-aaeb-d653ac318ef7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:46.194347 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f21b9ca-f22d-42c7-aaeb-d653ac318ef7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:46.195963 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f21b9ca-f22d-42c7-aaeb-d653ac318ef7 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:46.206552 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 917/1830] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:34:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:34:46.210710 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9631d7c-d7da-44a5-a158-884edee8b2c4 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:46.221958 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9631d7c-d7da-44a5-a158-884edee8b2c4 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:46.226763 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0391c792-04e8-4e92-b509-eee425e48c00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:46.227415 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0391c792-04e8-4e92-b509-eee425e48c00 admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:46.227696 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0391c792-04e8-4e92-b509-eee425e48c00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:46.227969 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0391c792-04e8-4e92-b509-eee425e48c00 admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:46.228610 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0391c792-04e8-4e92-b509-eee425e48c00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:46.230226 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0391c792-04e8-4e92-b509-eee425e48c00 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:46.241912 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 914/1831] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:34:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:34:46.248630 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f21b9ca-f22d-42c7-aaeb-d653ac318ef7 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:46.263740 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f21b9ca-f22d-42c7-aaeb-d653ac318ef7 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:46.274396 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9226bcab-0826-429c-bb1f-c1f018e05159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:46.275412 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9226bcab-0826-429c-bb1f-c1f018e05159 admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:46.275589 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9226bcab-0826-429c-bb1f-c1f018e05159 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:46.276066 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9226bcab-0826-429c-bb1f-c1f018e05159 admin admin] PATH_INFO: `/v3/projects/04d41af3bd244649a9bb8d6ee86fada2/users/b64c5593584746f5ac64a2314b925a59/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:46.294491 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9226bcab-0826-429c-bb1f-c1f018e05159 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04d41af3bd244649a9bb8d6ee86fada2, user_id=b64c5593584746f5ac64a2314b925a59, role_id=ca1a5695f8bd499aa0b8cedaeecd2cc6)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:46.298013 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9226bcab-0826-429c-bb1f-c1f018e05159 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:46.339260 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 918/1832] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:34:46 2022] PUT /identity/v3/projects/04d41af3bd244649a9bb8d6ee86fada2/users/b64c5593584746f5ac64a2314b925a59/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:34:46.349467 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71653fae-3f56-41aa-bd88-68f166c53fb2 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:46.349822 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71653fae-3f56-41aa-bd88-68f166c53fb2 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:46.350015 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71653fae-3f56-41aa-bd88-68f166c53fb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:46.391744 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-71653fae-3f56-41aa-bd88-68f166c53fb2 None None] MFA Rules not processed for user `b64c5593584746f5ac64a2314b925a59`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:34:46.393289 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-71653fae-3f56-41aa-bd88-68f166c53fb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:34:46.487361 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 915/1833] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:34:46 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:34:46.491781 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9226bcab-0826-429c-bb1f-c1f018e05159 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:46.520300 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:34:46.558344 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9226bcab-0826-429c-bb1f-c1f018e05159 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:46.563637 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027e878a-9236-44b1-9bfd-f7f89ea63f43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:46.564382 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027e878a-9236-44b1-9bfd-f7f89ea63f43 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:46.564662 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027e878a-9236-44b1-9bfd-f7f89ea63f43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:46.564931 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027e878a-9236-44b1-9bfd-f7f89ea63f43 admin admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:46.565549 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027e878a-9236-44b1-9bfd-f7f89ea63f43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:46.567497 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027e878a-9236-44b1-9bfd-f7f89ea63f43 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:46.594251 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 919/1834] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:34:46 2022] POST /identity/v3/projects => generated 383 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:34:46.598568 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71653fae-3f56-41aa-bd88-68f166c53fb2 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:46.609622 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71653fae-3f56-41aa-bd88-68f166c53fb2 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:46.667147 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450997c2-fbbf-4e1a-b97f-a524660d0a1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:46.667787 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450997c2-fbbf-4e1a-b97f-a524660d0a1d admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:46.668081 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450997c2-fbbf-4e1a-b97f-a524660d0a1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:46.668321 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450997c2-fbbf-4e1a-b97f-a524660d0a1d admin admin] PATH_INFO: `/v3/users` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:46.668956 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450997c2-fbbf-4e1a-b97f-a524660d0a1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:46.670874 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450997c2-fbbf-4e1a-b97f-a524660d0a1d admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:46.692867 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 916/1835] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:34:46 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:34:46.696801 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027e878a-9236-44b1-9bfd-f7f89ea63f43 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:46.708884 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027e878a-9236-44b1-9bfd-f7f89ea63f43 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:46.713637 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa722b0-b986-4b75-ac0f-0867152fa2ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:46.714250 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa722b0-b986-4b75-ac0f-0867152fa2ce admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:46.714528 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa722b0-b986-4b75-ac0f-0867152fa2ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:46.714814 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa722b0-b986-4b75-ac0f-0867152fa2ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:46.715468 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa722b0-b986-4b75-ac0f-0867152fa2ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:46.717063 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa722b0-b986-4b75-ac0f-0867152fa2ce admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:46.728555 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 920/1836] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:34:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:34:46.732983 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450997c2-fbbf-4e1a-b97f-a524660d0a1d admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:46.743669 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450997c2-fbbf-4e1a-b97f-a524660d0a1d admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:46.748503 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad98627-f4aa-4908-83cd-632a192af2a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:46.749201 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad98627-f4aa-4908-83cd-632a192af2a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:46.749430 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad98627-f4aa-4908-83cd-632a192af2a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:46.749693 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad98627-f4aa-4908-83cd-632a192af2a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:46.750327 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad98627-f4aa-4908-83cd-632a192af2a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:46.752502 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad98627-f4aa-4908-83cd-632a192af2a4 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:46.764860 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 917/1837] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:34:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:34:46.769076 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa722b0-b986-4b75-ac0f-0867152fa2ce admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:46.779448 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa722b0-b986-4b75-ac0f-0867152fa2ce admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:46.784754 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3a23f2-44a8-4507-975d-943d816d12b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:46.785401 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3a23f2-44a8-4507-975d-943d816d12b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:46.785678 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3a23f2-44a8-4507-975d-943d816d12b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:46.785948 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3a23f2-44a8-4507-975d-943d816d12b1 admin admin] PATH_INFO: `/v3/projects/82ef458a120c4d2db9b34026db750583/users/865ce72c936e45ff9821383d2054a7dc/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:46.801651 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3a23f2-44a8-4507-975d-943d816d12b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82ef458a120c4d2db9b34026db750583, user_id=865ce72c936e45ff9821383d2054a7dc, role_id=ca1a5695f8bd499aa0b8cedaeecd2cc6)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:46.804366 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3a23f2-44a8-4507-975d-943d816d12b1 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:46.817499 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 921/1838] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:34:46 2022] PUT /identity/v3/projects/82ef458a120c4d2db9b34026db750583/users/865ce72c936e45ff9821383d2054a7dc/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 31 12:34:46.827895 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a3e28a-8de9-4f61-a6be-ea266578cf19 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:46.828182 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a3e28a-8de9-4f61-a6be-ea266578cf19 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:46.828462 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a3e28a-8de9-4f61-a6be-ea266578cf19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:46.859094 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-33a3e28a-8de9-4f61-a6be-ea266578cf19 None None] MFA Rules not processed for user `865ce72c936e45ff9821383d2054a7dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:34:46.861087 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-33a3e28a-8de9-4f61-a6be-ea266578cf19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:34:46.891819 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:34:46.935631 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 918/1839] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:34:46 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:34:46.940233 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3a23f2-44a8-4507-975d-943d816d12b1 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:47.003459 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3a23f2-44a8-4507-975d-943d816d12b1 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:47.008403 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b1ad73-61c3-48bd-ae5c-e087d4cecd43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:47.009015 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b1ad73-61c3-48bd-ae5c-e087d4cecd43 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:47.009299 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b1ad73-61c3-48bd-ae5c-e087d4cecd43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:47.009571 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b1ad73-61c3-48bd-ae5c-e087d4cecd43 admin admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:47.010209 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b1ad73-61c3-48bd-ae5c-e087d4cecd43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:47.012134 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b1ad73-61c3-48bd-ae5c-e087d4cecd43 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:47.045047 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 922/1840] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:34:46 2022] POST /identity/v3/projects => generated 383 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:34:47.050714 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a3e28a-8de9-4f61-a6be-ea266578cf19 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:47.062644 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a3e28a-8de9-4f61-a6be-ea266578cf19 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:47.143585 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a094f22-a242-437f-8aef-be91cbc215c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:47.144070 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a094f22-a242-437f-8aef-be91cbc215c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:47.144345 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a094f22-a242-437f-8aef-be91cbc215c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:47.144621 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a094f22-a242-437f-8aef-be91cbc215c1 admin admin] PATH_INFO: `/v3/users` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:47.145353 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a094f22-a242-437f-8aef-be91cbc215c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:47.147977 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a094f22-a242-437f-8aef-be91cbc215c1 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:47.181445 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 919/1841] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:34:47 2022] POST /identity/v3/users => generated 298 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 31 12:34:47.182779 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b1ad73-61c3-48bd-ae5c-e087d4cecd43 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:47.193826 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b1ad73-61c3-48bd-ae5c-e087d4cecd43 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:47.211409 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981960b7-2994-499f-a643-6aca7dc4b944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:47.211409 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981960b7-2994-499f-a643-6aca7dc4b944 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:47.211409 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981960b7-2994-499f-a643-6aca7dc4b944 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:47.211409 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981960b7-2994-499f-a643-6aca7dc4b944 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:47.211822 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981960b7-2994-499f-a643-6aca7dc4b944 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:47.211822 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981960b7-2994-499f-a643-6aca7dc4b944 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:47.215010 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 923/1842] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:34:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:34:47.219350 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a094f22-a242-437f-8aef-be91cbc215c1 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:47.234352 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a094f22-a242-437f-8aef-be91cbc215c1 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:47.241856 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7e0f85-f387-4449-aac7-93ecd9ea329f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:47.242602 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7e0f85-f387-4449-aac7-93ecd9ea329f admin admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:47.242875 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7e0f85-f387-4449-aac7-93ecd9ea329f admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:47.243389 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7e0f85-f387-4449-aac7-93ecd9ea329f admin admin] PATH_INFO: `/v3/projects/9ee03b067cda461da79164cec2e2c624/users/d9b59a36bbd64954aa5c3f9e62f23f20/roles/176c07b4fe3b4537bdfeb4be21b379c5` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:47.260210 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7e0f85-f387-4449-aac7-93ecd9ea329f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ee03b067cda461da79164cec2e2c624, user_id=d9b59a36bbd64954aa5c3f9e62f23f20, role_id=176c07b4fe3b4537bdfeb4be21b379c5)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:47.263196 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7e0f85-f387-4449-aac7-93ecd9ea329f admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:47.281109 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 920/1843] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:34:47 2022] PUT /identity/v3/projects/9ee03b067cda461da79164cec2e2c624/users/d9b59a36bbd64954aa5c3f9e62f23f20/roles/176c07b4fe3b4537bdfeb4be21b379c5 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:34:47.286012 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981960b7-2994-499f-a643-6aca7dc4b944 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:47.301850 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981960b7-2994-499f-a643-6aca7dc4b944 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:47.413002 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:34:47.431739 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d056ec6-88d0-46dd-a798-3deac51ac5a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:47.432995 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d056ec6-88d0-46dd-a798-3deac51ac5a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:47.433493 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d056ec6-88d0-46dd-a798-3deac51ac5a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:47.433829 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d056ec6-88d0-46dd-a798-3deac51ac5a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:47.434573 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d056ec6-88d0-46dd-a798-3deac51ac5a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:47.436383 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d056ec6-88d0-46dd-a798-3deac51ac5a0 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:47.450791 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 924/1844] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:34:47 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:34:47.452428 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7e0f85-f387-4449-aac7-93ecd9ea329f admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:47.469332 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7e0f85-f387-4449-aac7-93ecd9ea329f admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:47.483737 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7b936e-56f9-452a-8976-ac2fbca01323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:47.484522 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7b936e-56f9-452a-8976-ac2fbca01323 admin admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:47.484931 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7b936e-56f9-452a-8976-ac2fbca01323 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:47.485297 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7b936e-56f9-452a-8976-ac2fbca01323 admin admin] PATH_INFO: `/v3/projects/9ee03b067cda461da79164cec2e2c624/users/d9b59a36bbd64954aa5c3f9e62f23f20/roles/176c07b4fe3b4537bdfeb4be21b379c5` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:47.488818 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7b936e-56f9-452a-8976-ac2fbca01323 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ee03b067cda461da79164cec2e2c624, user_id=d9b59a36bbd64954aa5c3f9e62f23f20, role_id=176c07b4fe3b4537bdfeb4be21b379c5)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:47.494674 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7b936e-56f9-452a-8976-ac2fbca01323 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:47.521878 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 921/1845] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:34:47 2022] PUT /identity/v3/projects/9ee03b067cda461da79164cec2e2c624/users/d9b59a36bbd64954aa5c3f9e62f23f20/roles/176c07b4fe3b4537bdfeb4be21b379c5 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:34:47.535818 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3c0b0d-e19b-40f3-9a1e-8907d3a5c7e4 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:47.536157 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3c0b0d-e19b-40f3-9a1e-8907d3a5c7e4 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:47.536400 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3c0b0d-e19b-40f3-9a1e-8907d3a5c7e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:47.573795 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-bc3c0b0d-e19b-40f3-9a1e-8907d3a5c7e4 None None] MFA Rules not processed for user `d9b59a36bbd64954aa5c3f9e62f23f20`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:34:47.575105 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-bc3c0b0d-e19b-40f3-9a1e-8907d3a5c7e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:34:47.665160 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 925/1846] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:34:47 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:34:47.693798 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7b936e-56f9-452a-8976-ac2fbca01323 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:47.695520 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-1c7b936e-56f9-452a-8976-ac2fbca01323 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:34:47.767011 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7b936e-56f9-452a-8976-ac2fbca01323 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:47.771107 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dbcf14-c007-487d-96cc-9a0118adbd04 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d6d7ea077d49e8a66c6c6762499702', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:47.771746 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dbcf14-c007-487d-96cc-9a0118adbd04 service cinder] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:47.772043 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dbcf14-c007-487d-96cc-9a0118adbd04 service cinder] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:47.772310 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dbcf14-c007-487d-96cc-9a0118adbd04 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:47.811965 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:34:47.841589 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dbcf14-c007-487d-96cc-9a0118adbd04 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:47.843415 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dbcf14-c007-487d-96cc-9a0118adbd04 service cinder] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:47.855480 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 922/1847] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:34:47 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:34:49.726176 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3c0b0d-e19b-40f3-9a1e-8907d3a5c7e4 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:49.736313 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3c0b0d-e19b-40f3-9a1e-8907d3a5c7e4 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:49.739921 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c890db8e-1bc7-4eb2-9b7d-4d594c813442 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d6d7ea077d49e8a66c6c6762499702', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:49.740557 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c890db8e-1bc7-4eb2-9b7d-4d594c813442 service cinder] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:49.740962 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c890db8e-1bc7-4eb2-9b7d-4d594c813442 service cinder] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:49.741435 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c890db8e-1bc7-4eb2-9b7d-4d594c813442 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:49.820824 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c890db8e-1bc7-4eb2-9b7d-4d594c813442 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:49.822280 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c890db8e-1bc7-4eb2-9b7d-4d594c813442 service cinder] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:49.831849 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 926/1848] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:34:49 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:34:52.104927 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dbcf14-c007-487d-96cc-9a0118adbd04 service cinder] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:52.113780 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dbcf14-c007-487d-96cc-9a0118adbd04 service cinder] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:52.117988 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410aeefa-afba-42ea-9008-65df957ff458 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d6d7ea077d49e8a66c6c6762499702', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:52.118605 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410aeefa-afba-42ea-9008-65df957ff458 service cinder] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:52.118904 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410aeefa-afba-42ea-9008-65df957ff458 service cinder] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:52.119256 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410aeefa-afba-42ea-9008-65df957ff458 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:52.129814 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410aeefa-afba-42ea-9008-65df957ff458 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:52.131248 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410aeefa-afba-42ea-9008-65df957ff458 service cinder] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:52.141261 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 923/1849] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:34:52 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:34:53.047692 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878dd3d6-3422-4afe-b931-b231e6343509 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:53.048627 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878dd3d6-3422-4afe-b931-b231e6343509 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:53.048909 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878dd3d6-3422-4afe-b931-b231e6343509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:53.072710 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-878dd3d6-3422-4afe-b931-b231e6343509 None None] MFA Rules not processed for user `bf4faacf7df248d28f32d7014dadcc07`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:34:53.074296 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-878dd3d6-3422-4afe-b931-b231e6343509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:34:53.135445 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 927/1850] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Oct 31 12:34:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:34:53.173929 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410aeefa-afba-42ea-9008-65df957ff458 service cinder] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:53.175383 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-410aeefa-afba-42ea-9008-65df957ff458 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:34:53.246651 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410aeefa-afba-42ea-9008-65df957ff458 service cinder] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:53.253441 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d6d332-f1a4-4581-acb4-b804a42cc5c8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2254a5786c249b4aa79244f7198ae65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:53.254200 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d6d332-f1a4-4581-acb4-b804a42cc5c8 service swift] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:53.254479 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d6d332-f1a4-4581-acb4-b804a42cc5c8 service swift] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:53.254935 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d6d332-f1a4-4581-acb4-b804a42cc5c8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:53.263470 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d6d332-f1a4-4581-acb4-b804a42cc5c8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:53.265067 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d6d332-f1a4-4581-acb4-b804a42cc5c8 service swift] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:53.274596 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 924/1851] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Oct 31 12:34:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 31 12:34:56.853850 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878dd3d6-3422-4afe-b931-b231e6343509 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:56.870755 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:34:56.930724 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878dd3d6-3422-4afe-b931-b231e6343509 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:56.937273 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7631ee1-5e6a-4415-a3d1-48e8519ce12c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:56.938478 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7631ee1-5e6a-4415-a3d1-48e8519ce12c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:56.938765 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7631ee1-5e6a-4415-a3d1-48e8519ce12c admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:56.939115 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7631ee1-5e6a-4415-a3d1-48e8519ce12c admin admin] PATH_INFO: `/v3/users/b64c5593584746f5ac64a2314b925a59` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:56.940639 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7631ee1-5e6a-4415-a3d1-48e8519ce12c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b64c5593584746f5ac64a2314b925a59)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:56.943354 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7631ee1-5e6a-4415-a3d1-48e8519ce12c admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:57.059181 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-c7631ee1-5e6a-4415-a3d1-48e8519ce12c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b64c5593584746f5ac64a2314b925a59'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:57.068236 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-c7631ee1-5e6a-4415-a3d1-48e8519ce12c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b64c5593584746f5ac64a2314b925a59'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:57.069021 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-c7631ee1-5e6a-4415-a3d1-48e8519ce12c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b64c5593584746f5ac64a2314b925a59'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:57.069726 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-c7631ee1-5e6a-4415-a3d1-48e8519ce12c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b64c5593584746f5ac64a2314b925a59'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:57.084043 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-c7631ee1-5e6a-4415-a3d1-48e8519ce12c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b64c5593584746f5ac64a2314b925a59'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:57.101209 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 928/1852] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:34:56 2022] DELETE /identity/v3/users/b64c5593584746f5ac64a2314b925a59 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:34:57.102477 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20fb485-eb2d-4ac5-a042-b146af4155f9 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:57.102985 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20fb485-eb2d-4ac5-a042-b146af4155f9 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:57.103077 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20fb485-eb2d-4ac5-a042-b146af4155f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:57.140267 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-b20fb485-eb2d-4ac5-a042-b146af4155f9 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:34:57.142392 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-b20fb485-eb2d-4ac5-a042-b146af4155f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:34:57.216551 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 925/1853] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:34:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:34:57.226434 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7631ee1-5e6a-4415-a3d1-48e8519ce12c admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:57.228299 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-c7631ee1-5e6a-4415-a3d1-48e8519ce12c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:34:57.311437 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7631ee1-5e6a-4415-a3d1-48e8519ce12c admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:57.316123 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386d7a54-b010-48ff-a110-d15a80f91beb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:57.316802 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386d7a54-b010-48ff-a110-d15a80f91beb service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:57.317162 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386d7a54-b010-48ff-a110-d15a80f91beb service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:57.317600 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386d7a54-b010-48ff-a110-d15a80f91beb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:57.328604 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386d7a54-b010-48ff-a110-d15a80f91beb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:57.332189 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386d7a54-b010-48ff-a110-d15a80f91beb service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:57.345525 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 929/1854] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:34:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:34:57.696687 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20fb485-eb2d-4ac5-a042-b146af4155f9 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:57.698491 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-b20fb485-eb2d-4ac5-a042-b146af4155f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:34:57.734317 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20fb485-eb2d-4ac5-a042-b146af4155f9 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:57.750115 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bea6ab9-b02f-4f1d-bc85-746f61c22b5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:57.752029 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bea6ab9-b02f-4f1d-bc85-746f61c22b5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:57.752311 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bea6ab9-b02f-4f1d-bc85-746f61c22b5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:57.754790 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bea6ab9-b02f-4f1d-bc85-746f61c22b5e admin admin] PATH_INFO: `/v3/projects/04d41af3bd244649a9bb8d6ee86fada2` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:57.757628 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bea6ab9-b02f-4f1d-bc85-746f61c22b5e admin admin] RBAC: Authorizing `identity:delete_project(project_id=04d41af3bd244649a9bb8d6ee86fada2)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:57.769057 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bea6ab9-b02f-4f1d-bc85-746f61c22b5e admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:57.840015 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-2bea6ab9-b02f-4f1d-bc85-746f61c22b5e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04d41af3bd244649a9bb8d6ee86fada2', 'request_id': 'req-2bea6ab9-b02f-4f1d-bc85-746f61c22b5e'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:57.841141 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:34:57.848750 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-2bea6ab9-b02f-4f1d-bc85-746f61c22b5e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04d41af3bd244649a9bb8d6ee86fada2', 'request_id': 'req-2bea6ab9-b02f-4f1d-bc85-746f61c22b5e'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:57.860935 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-2bea6ab9-b02f-4f1d-bc85-746f61c22b5e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04d41af3bd244649a9bb8d6ee86fada2', 'request_id': 'req-2bea6ab9-b02f-4f1d-bc85-746f61c22b5e'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:57.877533 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-2bea6ab9-b02f-4f1d-bc85-746f61c22b5e admin admin] The token cache is being invalidate because project 04d41af3bd244649a9bb8d6ee86fada2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:34:57.877913 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-2bea6ab9-b02f-4f1d-bc85-746f61c22b5e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:57.878626 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-2bea6ab9-b02f-4f1d-bc85-746f61c22b5e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:57.880586 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 926/1855] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:34:57 2022] DELETE /identity/v3/projects/04d41af3bd244649a9bb8d6ee86fada2 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:34:57.884758 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386d7a54-b010-48ff-a110-d15a80f91beb service neutron] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:57.905843 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386d7a54-b010-48ff-a110-d15a80f91beb service neutron] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:57.949069 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:34:57.987732 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db94e2e-34e0-4d54-8e19-c88e33e49b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:57.988531 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db94e2e-34e0-4d54-8e19-c88e33e49b42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:57.988865 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db94e2e-34e0-4d54-8e19-c88e33e49b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:57.989136 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db94e2e-34e0-4d54-8e19-c88e33e49b42 admin admin] PATH_INFO: `/v3/users/865ce72c936e45ff9821383d2054a7dc` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:57.990657 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db94e2e-34e0-4d54-8e19-c88e33e49b42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=865ce72c936e45ff9821383d2054a7dc)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:57.992925 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db94e2e-34e0-4d54-8e19-c88e33e49b42 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:58.058202 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-5db94e2e-34e0-4d54-8e19-c88e33e49b42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '865ce72c936e45ff9821383d2054a7dc'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:58.068860 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-5db94e2e-34e0-4d54-8e19-c88e33e49b42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '865ce72c936e45ff9821383d2054a7dc'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:58.069614 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-5db94e2e-34e0-4d54-8e19-c88e33e49b42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '865ce72c936e45ff9821383d2054a7dc'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:58.070533 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-5db94e2e-34e0-4d54-8e19-c88e33e49b42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '865ce72c936e45ff9821383d2054a7dc'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:58.091259 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-5db94e2e-34e0-4d54-8e19-c88e33e49b42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '865ce72c936e45ff9821383d2054a7dc'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:58.122103 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 930/1856] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:34:57 2022] DELETE /identity/v3/users/865ce72c936e45ff9821383d2054a7dc => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:34:58.586677 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bea6ab9-b02f-4f1d-bc85-746f61c22b5e admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:58.588820 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-2bea6ab9-b02f-4f1d-bc85-746f61c22b5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:34:58.604266 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bea6ab9-b02f-4f1d-bc85-746f61c22b5e admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:58.678280 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29c6a8a-cad9-4888-b347-0384ae9a7498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:58.678941 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29c6a8a-cad9-4888-b347-0384ae9a7498 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:58.679256 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29c6a8a-cad9-4888-b347-0384ae9a7498 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:58.679549 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29c6a8a-cad9-4888-b347-0384ae9a7498 admin admin] PATH_INFO: `/v3/projects/82ef458a120c4d2db9b34026db750583` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:58.680672 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29c6a8a-cad9-4888-b347-0384ae9a7498 admin admin] RBAC: Authorizing `identity:delete_project(project_id=82ef458a120c4d2db9b34026db750583)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:58.682921 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29c6a8a-cad9-4888-b347-0384ae9a7498 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:58.748263 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-f29c6a8a-cad9-4888-b347-0384ae9a7498 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82ef458a120c4d2db9b34026db750583', 'request_id': 'req-f29c6a8a-cad9-4888-b347-0384ae9a7498'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:58.755713 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-f29c6a8a-cad9-4888-b347-0384ae9a7498 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82ef458a120c4d2db9b34026db750583', 'request_id': 'req-f29c6a8a-cad9-4888-b347-0384ae9a7498'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:58.767793 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-f29c6a8a-cad9-4888-b347-0384ae9a7498 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82ef458a120c4d2db9b34026db750583', 'request_id': 'req-f29c6a8a-cad9-4888-b347-0384ae9a7498'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:58.783089 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-f29c6a8a-cad9-4888-b347-0384ae9a7498 admin admin] The token cache is being invalidate because project 82ef458a120c4d2db9b34026db750583 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:34:58.783408 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-f29c6a8a-cad9-4888-b347-0384ae9a7498 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:58.784069 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-f29c6a8a-cad9-4888-b347-0384ae9a7498 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:58.785750 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 927/1857] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:34:58 2022] DELETE /identity/v3/projects/82ef458a120c4d2db9b34026db750583 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:34:58.789527 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db94e2e-34e0-4d54-8e19-c88e33e49b42 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:58.790997 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-5db94e2e-34e0-4d54-8e19-c88e33e49b42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:34:58.804431 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db94e2e-34e0-4d54-8e19-c88e33e49b42 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:58.868236 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd33ed1-0aec-4b67-be88-e37728758945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:58.868893 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd33ed1-0aec-4b67-be88-e37728758945 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:58.869176 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd33ed1-0aec-4b67-be88-e37728758945 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:58.869448 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd33ed1-0aec-4b67-be88-e37728758945 admin admin] PATH_INFO: `/v3/users/d9b59a36bbd64954aa5c3f9e62f23f20` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:58.870713 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd33ed1-0aec-4b67-be88-e37728758945 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9b59a36bbd64954aa5c3f9e62f23f20)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:58.873010 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd33ed1-0aec-4b67-be88-e37728758945 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:58.927189 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-5dd33ed1-0aec-4b67-be88-e37728758945 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9b59a36bbd64954aa5c3f9e62f23f20'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:58.934542 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-5dd33ed1-0aec-4b67-be88-e37728758945 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9b59a36bbd64954aa5c3f9e62f23f20'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:58.935320 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-5dd33ed1-0aec-4b67-be88-e37728758945 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9b59a36bbd64954aa5c3f9e62f23f20'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:58.936048 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-5dd33ed1-0aec-4b67-be88-e37728758945 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9b59a36bbd64954aa5c3f9e62f23f20'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:58.938981 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:34:58.954310 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-5dd33ed1-0aec-4b67-be88-e37728758945 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9b59a36bbd64954aa5c3f9e62f23f20'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:58.966891 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 931/1858] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:34:58 2022] DELETE /identity/v3/users/d9b59a36bbd64954aa5c3f9e62f23f20 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:34:59.295914 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29c6a8a-cad9-4888-b347-0384ae9a7498 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:59.298900 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-f29c6a8a-cad9-4888-b347-0384ae9a7498 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:34:59.316398 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29c6a8a-cad9-4888-b347-0384ae9a7498 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:59.379618 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818adb36-8edf-4538-b463-e611a676d879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:59.380276 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818adb36-8edf-4538-b463-e611a676d879 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:59.380559 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818adb36-8edf-4538-b463-e611a676d879 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:59.380842 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818adb36-8edf-4538-b463-e611a676d879 admin admin] PATH_INFO: `/v3/projects/9ee03b067cda461da79164cec2e2c624` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:59.382611 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818adb36-8edf-4538-b463-e611a676d879 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ee03b067cda461da79164cec2e2c624)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:59.384762 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818adb36-8edf-4538-b463-e611a676d879 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:59.397201 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:34:59.451607 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-818adb36-8edf-4538-b463-e611a676d879 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ee03b067cda461da79164cec2e2c624', 'request_id': 'req-818adb36-8edf-4538-b463-e611a676d879'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:59.457609 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-818adb36-8edf-4538-b463-e611a676d879 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ee03b067cda461da79164cec2e2c624', 'request_id': 'req-818adb36-8edf-4538-b463-e611a676d879'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:59.468096 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-818adb36-8edf-4538-b463-e611a676d879 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ee03b067cda461da79164cec2e2c624', 'request_id': 'req-818adb36-8edf-4538-b463-e611a676d879'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:59.483561 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-818adb36-8edf-4538-b463-e611a676d879 admin admin] The token cache is being invalidate because project 9ee03b067cda461da79164cec2e2c624 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:34:59.483882 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-818adb36-8edf-4538-b463-e611a676d879 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:59.484660 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-818adb36-8edf-4538-b463-e611a676d879 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:34:59.491687 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 928/1859] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:34:59 2022] DELETE /identity/v3/projects/9ee03b067cda461da79164cec2e2c624 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:34:59.492673 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d453cb4-17c4-4892-bd5c-34d899caad1b None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:59.493139 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d453cb4-17c4-4892-bd5c-34d899caad1b None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:59.493936 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d453cb4-17c4-4892-bd5c-34d899caad1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:59.515765 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-9d453cb4-17c4-4892-bd5c-34d899caad1b None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:34:59.517345 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-9d453cb4-17c4-4892-bd5c-34d899caad1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:34:59.591195 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 932/1860] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:34:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 31 12:34:59.597813 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9735a96c-5a22-452d-b65f-7e6f35b9410b None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:59.598337 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9735a96c-5a22-452d-b65f-7e6f35b9410b None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:59.598490 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9735a96c-5a22-452d-b65f-7e6f35b9410b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:59.624346 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-9735a96c-5a22-452d-b65f-7e6f35b9410b None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:34:59.626150 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-9735a96c-5a22-452d-b65f-7e6f35b9410b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:34:59.638796 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 929/1861] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:34:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:34:59.639705 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d453cb4-17c4-4892-bd5c-34d899caad1b None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:59.650865 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d453cb4-17c4-4892-bd5c-34d899caad1b None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:59.658757 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec40321-8e8b-48bc-ae5b-03df4505f83b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:59.659424 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec40321-8e8b-48bc-ae5b-03df4505f83b admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:59.659713 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec40321-8e8b-48bc-ae5b-03df4505f83b admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:59.659977 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec40321-8e8b-48bc-ae5b-03df4505f83b admin admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:59.660624 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec40321-8e8b-48bc-ae5b-03df4505f83b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:34:59.661217 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec40321-8e8b-48bc-ae5b-03df4505f83b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:59.663021 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec40321-8e8b-48bc-ae5b-03df4505f83b admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:59.678408 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 933/1862] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Oct 31 12:34:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 31 12:34:59.685185 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9735a96c-5a22-452d-b65f-7e6f35b9410b None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:59.695627 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9735a96c-5a22-452d-b65f-7e6f35b9410b None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:59.700886 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f63b89-4dbd-4d84-9428-6850921e8bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:59.701490 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f63b89-4dbd-4d84-9428-6850921e8bed admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:59.701766 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f63b89-4dbd-4d84-9428-6850921e8bed admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:59.702030 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f63b89-4dbd-4d84-9428-6850921e8bed admin admin] PATH_INFO: `/v3/projects` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:59.702763 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f63b89-4dbd-4d84-9428-6850921e8bed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:59.705472 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f63b89-4dbd-4d84-9428-6850921e8bed admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:59.749337 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 930/1863] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:34:59 2022] POST /identity/v3/projects => generated 373 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:34:59.750766 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec40321-8e8b-48bc-ae5b-03df4505f83b admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:59.763013 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec40321-8e8b-48bc-ae5b-03df4505f83b admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:59.847703 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc356046-9cf0-4420-95d0-72740a989590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:59.848313 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc356046-9cf0-4420-95d0-72740a989590 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:59.848585 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc356046-9cf0-4420-95d0-72740a989590 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:59.848854 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc356046-9cf0-4420-95d0-72740a989590 admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:59.849574 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc356046-9cf0-4420-95d0-72740a989590 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:59.851557 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc356046-9cf0-4420-95d0-72740a989590 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:59.875389 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 934/1864] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:34:59 2022] POST /identity/v3/users => generated 287 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:34:59.879422 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f63b89-4dbd-4d84-9428-6850921e8bed admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:59.891487 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f63b89-4dbd-4d84-9428-6850921e8bed admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:59.896709 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f26b278-1391-4d0c-be49-2ff2f7fd9400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:59.897463 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f26b278-1391-4d0c-be49-2ff2f7fd9400 admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:59.897920 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f26b278-1391-4d0c-be49-2ff2f7fd9400 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:59.898216 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f26b278-1391-4d0c-be49-2ff2f7fd9400 admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:59.898848 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f26b278-1391-4d0c-be49-2ff2f7fd9400 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:59.900659 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f26b278-1391-4d0c-be49-2ff2f7fd9400 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:59.915223 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 931/1865] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:34:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:34:59.917051 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc356046-9cf0-4420-95d0-72740a989590 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:59.928073 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc356046-9cf0-4420-95d0-72740a989590 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:59.932903 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7299ae-2ccc-46ac-a3ec-d529cd6eb8b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:59.933613 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7299ae-2ccc-46ac-a3ec-d529cd6eb8b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:59.934060 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7299ae-2ccc-46ac-a3ec-d529cd6eb8b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:59.934234 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7299ae-2ccc-46ac-a3ec-d529cd6eb8b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:59.934917 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7299ae-2ccc-46ac-a3ec-d529cd6eb8b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:59.936542 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7299ae-2ccc-46ac-a3ec-d529cd6eb8b2 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:34:59.951311 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 935/1866] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:34:59 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:34:59.952144 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f26b278-1391-4d0c-be49-2ff2f7fd9400 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:34:59.964636 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f26b278-1391-4d0c-be49-2ff2f7fd9400 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:34:59.970350 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed8f6d8-05ce-4b4e-9226-c95b29c288b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:34:59.972294 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed8f6d8-05ce-4b4e-9226-c95b29c288b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:34:59.972786 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed8f6d8-05ce-4b4e-9226-c95b29c288b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:34:59.973169 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed8f6d8-05ce-4b4e-9226-c95b29c288b8 admin admin] PATH_INFO: `/v3/projects/4668b52a7eeb4a57a4ba95c2c640df44/users/286ecc31b7f4469384e754db1cf595ae/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:34:59.990919 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed8f6d8-05ce-4b4e-9226-c95b29c288b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4668b52a7eeb4a57a4ba95c2c640df44, user_id=286ecc31b7f4469384e754db1cf595ae, role_id=ca1a5695f8bd499aa0b8cedaeecd2cc6)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:34:59.993864 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed8f6d8-05ce-4b4e-9226-c95b29c288b8 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:35:00.009229 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 932/1867] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:34:59 2022] PUT /identity/v3/projects/4668b52a7eeb4a57a4ba95c2c640df44/users/286ecc31b7f4469384e754db1cf595ae/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 31 12:35:00.020264 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261fbf29-1b0a-44e2-bab3-fcb8da21a7ee None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:35:00.020885 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261fbf29-1b0a-44e2-bab3-fcb8da21a7ee None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:35:00.021222 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261fbf29-1b0a-44e2-bab3-fcb8da21a7ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:35:00.070473 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-261fbf29-1b0a-44e2-bab3-fcb8da21a7ee None None] MFA Rules not processed for user `286ecc31b7f4469384e754db1cf595ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:35:00.071835 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-261fbf29-1b0a-44e2-bab3-fcb8da21a7ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:35:00.107989 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:35:00.153152 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 936/1868] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:35:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:35:00.163798 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed8f6d8-05ce-4b4e-9226-c95b29c288b8 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:35:00.165502 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-aed8f6d8-05ce-4b4e-9226-c95b29c288b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:35:00.247688 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed8f6d8-05ce-4b4e-9226-c95b29c288b8 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:35:00.253967 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41bb047-2721-49cf-9234-50ca154c6534 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d6d7ea077d49e8a66c6c6762499702', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:35:00.254766 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41bb047-2721-49cf-9234-50ca154c6534 service cinder] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:35:00.254945 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41bb047-2721-49cf-9234-50ca154c6534 service cinder] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:35:00.255830 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41bb047-2721-49cf-9234-50ca154c6534 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:35:00.264917 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41bb047-2721-49cf-9234-50ca154c6534 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:35:00.266989 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41bb047-2721-49cf-9234-50ca154c6534 service cinder] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:35:00.279449 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 933/1869] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:35:00 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 31 12:35:06.456444 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261fbf29-1b0a-44e2-bab3-fcb8da21a7ee None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:35:06.457941 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-261fbf29-1b0a-44e2-bab3-fcb8da21a7ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:35:06.553725 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261fbf29-1b0a-44e2-bab3-fcb8da21a7ee None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:35:06.557797 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b15f75-666a-43a0-aebc-225de57e4fc2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:35:06.558725 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b15f75-666a-43a0-aebc-225de57e4fc2 service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:35:06.559151 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b15f75-666a-43a0-aebc-225de57e4fc2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:35:06.559609 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b15f75-666a-43a0-aebc-225de57e4fc2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:35:06.561781 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-d3b15f75-666a-43a0-aebc-225de57e4fc2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:35:06.665880 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b15f75-666a-43a0-aebc-225de57e4fc2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:35:06.667759 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b15f75-666a-43a0-aebc-225de57e4fc2 service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:35:06.710813 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 937/1870] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:35:06 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:35:10.272664 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=73,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:35:11.584470 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41bb047-2721-49cf-9234-50ca154c6534 service cinder] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:35:11.665258 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41bb047-2721-49cf-9234-50ca154c6534 service cinder] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:35:11.670238 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4efa79-2d9d-45a6-ada5-aa63bd09bcfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:35:11.670864 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4efa79-2d9d-45a6-ada5-aa63bd09bcfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:35:11.671139 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4efa79-2d9d-45a6-ada5-aa63bd09bcfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:35:11.671844 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4efa79-2d9d-45a6-ada5-aa63bd09bcfe admin admin] PATH_INFO: `/v3/users/286ecc31b7f4469384e754db1cf595ae` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:35:11.672950 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4efa79-2d9d-45a6-ada5-aa63bd09bcfe admin admin] RBAC: Authorizing `identity:delete_user(user_id=286ecc31b7f4469384e754db1cf595ae)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:35:11.674870 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4efa79-2d9d-45a6-ada5-aa63bd09bcfe admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:35:11.726791 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-2f4efa79-2d9d-45a6-ada5-aa63bd09bcfe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '286ecc31b7f4469384e754db1cf595ae'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:35:11.734724 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-2f4efa79-2d9d-45a6-ada5-aa63bd09bcfe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '286ecc31b7f4469384e754db1cf595ae'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:35:11.735491 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-2f4efa79-2d9d-45a6-ada5-aa63bd09bcfe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '286ecc31b7f4469384e754db1cf595ae'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:35:11.754083 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-2f4efa79-2d9d-45a6-ada5-aa63bd09bcfe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '286ecc31b7f4469384e754db1cf595ae'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:35:11.763851 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-2f4efa79-2d9d-45a6-ada5-aa63bd09bcfe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '286ecc31b7f4469384e754db1cf595ae'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:35:11.774574 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 934/1871] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:35:11 2022] DELETE /identity/v3/users/286ecc31b7f4469384e754db1cf595ae => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:35:11.778980 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9e71b6-212b-406f-aa69-38cf2c14a60d None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:35:11.779516 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9e71b6-212b-406f-aa69-38cf2c14a60d None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:35:11.779693 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9e71b6-212b-406f-aa69-38cf2c14a60d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:35:11.800259 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-8d9e71b6-212b-406f-aa69-38cf2c14a60d None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:35:11.801526 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-8d9e71b6-212b-406f-aa69-38cf2c14a60d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:35:11.886968 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 938/1872] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:35:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:35:11.897499 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4efa79-2d9d-45a6-ada5-aa63bd09bcfe admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:35:11.898975 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-2f4efa79-2d9d-45a6-ada5-aa63bd09bcfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:35:11.963395 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4efa79-2d9d-45a6-ada5-aa63bd09bcfe admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:35:11.967606 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3341b597-a22b-4321-a2c4-df11b1de3d74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:35:11.968094 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3341b597-a22b-4321-a2c4-df11b1de3d74 service neutron] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:35:11.968429 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3341b597-a22b-4321-a2c4-df11b1de3d74 service neutron] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:35:11.969204 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3341b597-a22b-4321-a2c4-df11b1de3d74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:35:11.978770 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3341b597-a22b-4321-a2c4-df11b1de3d74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:35:11.980401 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3341b597-a22b-4321-a2c4-df11b1de3d74 service neutron] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:35:11.990524 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 935/1873] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:35:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:35:12.416517 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9e71b6-212b-406f-aa69-38cf2c14a60d None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:35:12.417916 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-8d9e71b6-212b-406f-aa69-38cf2c14a60d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:35:12.439443 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9e71b6-212b-406f-aa69-38cf2c14a60d None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:35:12.445826 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaeb61e-37c7-4218-8ca8-ff2c649f06e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:35:12.446684 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaeb61e-37c7-4218-8ca8-ff2c649f06e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:35:12.447112 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaeb61e-37c7-4218-8ca8-ff2c649f06e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:35:12.447499 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaeb61e-37c7-4218-8ca8-ff2c649f06e9 admin admin] PATH_INFO: `/v3/projects/4668b52a7eeb4a57a4ba95c2c640df44` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:35:12.449176 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaeb61e-37c7-4218-8ca8-ff2c649f06e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4668b52a7eeb4a57a4ba95c2c640df44)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:35:12.451355 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaeb61e-37c7-4218-8ca8-ff2c649f06e9 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:35:12.472817 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:35:12.508964 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-bdaeb61e-37c7-4218-8ca8-ff2c649f06e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4668b52a7eeb4a57a4ba95c2c640df44', 'request_id': 'req-bdaeb61e-37c7-4218-8ca8-ff2c649f06e9'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:35:12.516425 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-bdaeb61e-37c7-4218-8ca8-ff2c649f06e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4668b52a7eeb4a57a4ba95c2c640df44', 'request_id': 'req-bdaeb61e-37c7-4218-8ca8-ff2c649f06e9'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:35:12.525452 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-bdaeb61e-37c7-4218-8ca8-ff2c649f06e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4668b52a7eeb4a57a4ba95c2c640df44', 'request_id': 'req-bdaeb61e-37c7-4218-8ca8-ff2c649f06e9'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:35:12.538291 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-bdaeb61e-37c7-4218-8ca8-ff2c649f06e9 admin admin] The token cache is being invalidate because project 4668b52a7eeb4a57a4ba95c2c640df44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:35:12.538577 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-bdaeb61e-37c7-4218-8ca8-ff2c649f06e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:35:12.539688 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-bdaeb61e-37c7-4218-8ca8-ff2c649f06e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:35:12.541553 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 939/1874] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:35:12 2022] DELETE /identity/v3/projects/4668b52a7eeb4a57a4ba95c2c640df44 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 31 12:35:12.546569 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c39801-046c-4961-a201-5c9efe17b783 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:35:12.546857 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c39801-046c-4961-a201-5c9efe17b783 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:35:12.547134 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c39801-046c-4961-a201-5c9efe17b783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:35:12.568714 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-e1c39801-046c-4961-a201-5c9efe17b783 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:35:12.569930 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-e1c39801-046c-4961-a201-5c9efe17b783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:35:12.591386 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:35:12.646126 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 936/1875] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:35:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:35:12.654052 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdb2350-c488-46ef-b1b8-e29a9a5e5e41 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:35:12.654333 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdb2350-c488-46ef-b1b8-e29a9a5e5e41 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:35:12.654617 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdb2350-c488-46ef-b1b8-e29a9a5e5e41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:35:12.679102 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-dfdb2350-c488-46ef-b1b8-e29a9a5e5e41 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:35:12.680371 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-dfdb2350-c488-46ef-b1b8-e29a9a5e5e41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:35:12.689921 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 940/1876] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:35:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:35:12.694403 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c39801-046c-4961-a201-5c9efe17b783 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:35:12.708880 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c39801-046c-4961-a201-5c9efe17b783 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:35:12.715589 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7fbcb3-f969-4864-bc2b-165f6ef95b00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:35:12.716727 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7fbcb3-f969-4864-bc2b-165f6ef95b00 admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:35:12.717118 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7fbcb3-f969-4864-bc2b-165f6ef95b00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:35:12.717481 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7fbcb3-f969-4864-bc2b-165f6ef95b00 admin admin] PATH_INFO: `/v3/domains` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:35:12.718616 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7fbcb3-f969-4864-bc2b-165f6ef95b00 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:35:12.719150 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7fbcb3-f969-4864-bc2b-165f6ef95b00 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:35:12.721113 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7fbcb3-f969-4864-bc2b-165f6ef95b00 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:35:12.745326 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 937/1877] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Oct 31 12:35:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:35:12.746826 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdb2350-c488-46ef-b1b8-e29a9a5e5e41 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:35:12.763420 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdb2350-c488-46ef-b1b8-e29a9a5e5e41 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:35:12.773236 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869856be-51c4-41ca-8cac-55a125f23204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:35:12.774249 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869856be-51c4-41ca-8cac-55a125f23204 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:35:12.774876 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869856be-51c4-41ca-8cac-55a125f23204 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:35:12.775505 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869856be-51c4-41ca-8cac-55a125f23204 admin admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:35:12.776303 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869856be-51c4-41ca-8cac-55a125f23204 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:35:12.779975 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869856be-51c4-41ca-8cac-55a125f23204 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:35:12.822275 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 941/1878] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:35:12 2022] POST /identity/v3/projects => generated 391 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:35:12.823510 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7fbcb3-f969-4864-bc2b-165f6ef95b00 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:35:12.835559 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7fbcb3-f969-4864-bc2b-165f6ef95b00 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:35:12.957420 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b13091f-2d29-4878-a56d-1e030b243995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:35:12.958829 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b13091f-2d29-4878-a56d-1e030b243995 admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:35:12.959306 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b13091f-2d29-4878-a56d-1e030b243995 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:35:12.959739 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b13091f-2d29-4878-a56d-1e030b243995 admin admin] PATH_INFO: `/v3/users` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:35:12.963298 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b13091f-2d29-4878-a56d-1e030b243995 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:35:12.965247 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b13091f-2d29-4878-a56d-1e030b243995 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:35:13.001910 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 938/1879] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:35:12 2022] POST /identity/v3/users => generated 296 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 31 12:35:13.005979 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869856be-51c4-41ca-8cac-55a125f23204 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:35:13.018960 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869856be-51c4-41ca-8cac-55a125f23204 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:35:13.032784 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d76aae-ae1e-46fc-805a-11df61bff6ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:35:13.033956 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d76aae-ae1e-46fc-805a-11df61bff6ec admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:35:13.034242 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d76aae-ae1e-46fc-805a-11df61bff6ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:35:13.034619 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d76aae-ae1e-46fc-805a-11df61bff6ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:35:13.035436 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d76aae-ae1e-46fc-805a-11df61bff6ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:35:13.037948 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d76aae-ae1e-46fc-805a-11df61bff6ec admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:35:13.057210 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 942/1880] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:35:13 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:35:13.063585 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b13091f-2d29-4878-a56d-1e030b243995 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:35:13.086537 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b13091f-2d29-4878-a56d-1e030b243995 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:35:13.106150 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70e151b-89be-47e3-ae2d-e57f58fa813b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:35:13.107222 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70e151b-89be-47e3-ae2d-e57f58fa813b admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:35:13.107505 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70e151b-89be-47e3-ae2d-e57f58fa813b admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:35:13.108109 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70e151b-89be-47e3-ae2d-e57f58fa813b admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:35:13.108416 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70e151b-89be-47e3-ae2d-e57f58fa813b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:35:13.115241 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70e151b-89be-47e3-ae2d-e57f58fa813b admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:35:13.133958 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 939/1881] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:35:13 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:35:13.135360 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d76aae-ae1e-46fc-805a-11df61bff6ec admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:35:13.194791 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d76aae-ae1e-46fc-805a-11df61bff6ec admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:35:13.216287 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a50be9-32cd-4dfd-b665-adc6247f648c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:35:13.216577 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a50be9-32cd-4dfd-b665-adc6247f648c admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:35:13.216880 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a50be9-32cd-4dfd-b665-adc6247f648c admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:35:13.217151 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a50be9-32cd-4dfd-b665-adc6247f648c admin admin] PATH_INFO: `/v3/projects/1985e64225234725930ce72abeac9ed4/users/356fce5f98ab43be8bbbca23fbbaa02d/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:35:13.267328 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a50be9-32cd-4dfd-b665-adc6247f648c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1985e64225234725930ce72abeac9ed4, user_id=356fce5f98ab43be8bbbca23fbbaa02d, role_id=ca1a5695f8bd499aa0b8cedaeecd2cc6)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:35:13.271297 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a50be9-32cd-4dfd-b665-adc6247f648c admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:35:13.296280 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 943/1882] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:35:13 2022] PUT /identity/v3/projects/1985e64225234725930ce72abeac9ed4/users/356fce5f98ab43be8bbbca23fbbaa02d/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 31 12:35:13.308394 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513b2519-9627-45d0-8cb1-8079cbedbc7a None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:35:13.308750 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513b2519-9627-45d0-8cb1-8079cbedbc7a None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:35:13.309089 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513b2519-9627-45d0-8cb1-8079cbedbc7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:35:13.373825 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-513b2519-9627-45d0-8cb1-8079cbedbc7a None None] MFA Rules not processed for user `356fce5f98ab43be8bbbca23fbbaa02d`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:35:13.376162 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-513b2519-9627-45d0-8cb1-8079cbedbc7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:35:13.494482 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 940/1883] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:35:13 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:35:13.506133 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a50be9-32cd-4dfd-b665-adc6247f648c admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:35:13.508216 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-e3a50be9-32cd-4dfd-b665-adc6247f648c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:35:13.639690 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a50be9-32cd-4dfd-b665-adc6247f648c admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:35:13.643987 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3500fb6-a6e1-4ed4-be40-bb0f037e59d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d6d7ea077d49e8a66c6c6762499702', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:35:13.644823 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3500fb6-a6e1-4ed4-be40-bb0f037e59d8 service cinder] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:35:13.645546 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3500fb6-a6e1-4ed4-be40-bb0f037e59d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:35:13.646021 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3500fb6-a6e1-4ed4-be40-bb0f037e59d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:35:13.656043 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3500fb6-a6e1-4ed4-be40-bb0f037e59d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:35:13.658575 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3500fb6-a6e1-4ed4-be40-bb0f037e59d8 service cinder] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:35:13.671352 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 944/1884] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:35:13 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:35:20.190161 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682799fe-035d-4eb8-8c10-823c017c6949 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:35:20.190644 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682799fe-035d-4eb8-8c10-823c017c6949 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:35:20.190856 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682799fe-035d-4eb8-8c10-823c017c6949 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:35:20.209967 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-682799fe-035d-4eb8-8c10-823c017c6949 None None] MFA Rules not processed for user `bf4faacf7df248d28f32d7014dadcc07`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:35:20.211193 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-682799fe-035d-4eb8-8c10-823c017c6949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:35:20.224205 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:35:20.273664 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 941/1885] 10.222.0.15 () {58 vars in 967 bytes} [Mon Oct 31 12:35:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:35:20.286526 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3500fb6-a6e1-4ed4-be40-bb0f037e59d8 service cinder] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:35:20.287922 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-b3500fb6-a6e1-4ed4-be40-bb0f037e59d8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:35:20.340831 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:35:20.370744 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3500fb6-a6e1-4ed4-be40-bb0f037e59d8 service cinder] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:35:20.376749 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cded56-08ea-41db-b6f9-259fc3587ec4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2254a5786c249b4aa79244f7198ae65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:35:20.377366 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cded56-08ea-41db-b6f9-259fc3587ec4 service swift] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:35:20.377646 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cded56-08ea-41db-b6f9-259fc3587ec4 service swift] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:35:20.377924 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cded56-08ea-41db-b6f9-259fc3587ec4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:35:20.387672 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cded56-08ea-41db-b6f9-259fc3587ec4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:35:20.389256 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cded56-08ea-41db-b6f9-259fc3587ec4 service swift] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:35:20.399741 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 945/1886] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Oct 31 12:35:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 31 12:35:25.777263 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682799fe-035d-4eb8-8c10-823c017c6949 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:35:25.778711 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-682799fe-035d-4eb8-8c10-823c017c6949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:35:25.853293 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682799fe-035d-4eb8-8c10-823c017c6949 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:35:25.857857 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b5e7e1-bf18-4dbb-a963-c3ea2808a88d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:35:25.858491 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b5e7e1-bf18-4dbb-a963-c3ea2808a88d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:35:25.858764 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b5e7e1-bf18-4dbb-a963-c3ea2808a88d admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:35:25.859035 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b5e7e1-bf18-4dbb-a963-c3ea2808a88d admin admin] PATH_INFO: `/v3/users/1918cb60ccd04e909b78dca9b0166be9` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:35:25.860879 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b5e7e1-bf18-4dbb-a963-c3ea2808a88d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1918cb60ccd04e909b78dca9b0166be9)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:35:25.862695 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b5e7e1-bf18-4dbb-a963-c3ea2808a88d admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:35:25.918631 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-b2b5e7e1-bf18-4dbb-a963-c3ea2808a88d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1918cb60ccd04e909b78dca9b0166be9'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:35:25.926322 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-b2b5e7e1-bf18-4dbb-a963-c3ea2808a88d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1918cb60ccd04e909b78dca9b0166be9'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:35:25.926946 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-b2b5e7e1-bf18-4dbb-a963-c3ea2808a88d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1918cb60ccd04e909b78dca9b0166be9'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:35:25.939572 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-b2b5e7e1-bf18-4dbb-a963-c3ea2808a88d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1918cb60ccd04e909b78dca9b0166be9'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:35:25.946403 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-b2b5e7e1-bf18-4dbb-a963-c3ea2808a88d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1918cb60ccd04e909b78dca9b0166be9'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:35:25.952346 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 942/1887] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:35:25 2022] DELETE /identity/v3/users/1918cb60ccd04e909b78dca9b0166be9 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:35:26.158172 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cded56-08ea-41db-b6f9-259fc3587ec4 service swift] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:35:26.168783 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cded56-08ea-41db-b6f9-259fc3587ec4 service swift] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:35:26.233653 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112da0c3-fc9e-4bd2-81e4-4a4c81178cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:35:26.234295 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112da0c3-fc9e-4bd2-81e4-4a4c81178cc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:35:26.234567 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112da0c3-fc9e-4bd2-81e4-4a4c81178cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:35:26.234890 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112da0c3-fc9e-4bd2-81e4-4a4c81178cc9 admin admin] PATH_INFO: `/v3/projects/4fe4270c111747be8d61e4ae993bd7ba` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:35:26.236057 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112da0c3-fc9e-4bd2-81e4-4a4c81178cc9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4fe4270c111747be8d61e4ae993bd7ba)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:35:26.237895 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112da0c3-fc9e-4bd2-81e4-4a4c81178cc9 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:35:26.283434 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-112da0c3-fc9e-4bd2-81e4-4a4c81178cc9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4fe4270c111747be8d61e4ae993bd7ba', 'request_id': 'req-112da0c3-fc9e-4bd2-81e4-4a4c81178cc9'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:35:26.289416 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-112da0c3-fc9e-4bd2-81e4-4a4c81178cc9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4fe4270c111747be8d61e4ae993bd7ba', 'request_id': 'req-112da0c3-fc9e-4bd2-81e4-4a4c81178cc9'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:35:26.296446 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-112da0c3-fc9e-4bd2-81e4-4a4c81178cc9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4fe4270c111747be8d61e4ae993bd7ba', 'request_id': 'req-112da0c3-fc9e-4bd2-81e4-4a4c81178cc9'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:35:26.307045 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-112da0c3-fc9e-4bd2-81e4-4a4c81178cc9 admin admin] The token cache is being invalidate because project 4fe4270c111747be8d61e4ae993bd7ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:35:26.307369 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-112da0c3-fc9e-4bd2-81e4-4a4c81178cc9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:35:26.307983 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-112da0c3-fc9e-4bd2-81e4-4a4c81178cc9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:35:26.309446 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 946/1888] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:35:26 2022] DELETE /identity/v3/projects/4fe4270c111747be8d61e4ae993bd7ba => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:35:31.941026 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b5e7e1-bf18-4dbb-a963-c3ea2808a88d admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:35:31.943188 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-b2b5e7e1-bf18-4dbb-a963-c3ea2808a88d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:35:31.957822 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b5e7e1-bf18-4dbb-a963-c3ea2808a88d admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:35:32.027105 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:35:32.043599 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0b2123-f9b5-4336-a76b-6e7cf7d70ab2 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:35:32.044484 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0b2123-f9b5-4336-a76b-6e7cf7d70ab2 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:35:32.044988 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0b2123-f9b5-4336-a76b-6e7cf7d70ab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:35:32.057297 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1495fd13-f2b6-48b8-862e-6831253b462a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:35:32.057941 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1495fd13-f2b6-48b8-862e-6831253b462a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:35:32.058218 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1495fd13-f2b6-48b8-862e-6831253b462a admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:35:32.058489 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1495fd13-f2b6-48b8-862e-6831253b462a admin admin] PATH_INFO: `/v3/users/6ab36e845b514205b6a652827dab9d76` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:35:32.059643 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1495fd13-f2b6-48b8-862e-6831253b462a admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ab36e845b514205b6a652827dab9d76)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:35:32.061558 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1495fd13-f2b6-48b8-862e-6831253b462a admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:35:32.066004 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-6b0b2123-f9b5-4336-a76b-6e7cf7d70ab2 None None] MFA Rules not processed for user `b2c03f0365ee4a26b8c076ae365325b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:35:32.067330 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-6b0b2123-f9b5-4336-a76b-6e7cf7d70ab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:35:32.120614 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-1495fd13-f2b6-48b8-862e-6831253b462a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ab36e845b514205b6a652827dab9d76'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:35:32.129837 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-1495fd13-f2b6-48b8-862e-6831253b462a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ab36e845b514205b6a652827dab9d76'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:35:32.130492 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-1495fd13-f2b6-48b8-862e-6831253b462a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ab36e845b514205b6a652827dab9d76'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:35:32.136976 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 947/1889] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:35:32 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:35:32.143862 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-1495fd13-f2b6-48b8-862e-6831253b462a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ab36e845b514205b6a652827dab9d76'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:35:32.157167 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-1495fd13-f2b6-48b8-862e-6831253b462a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ab36e845b514205b6a652827dab9d76'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:35:32.164558 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 943/1890] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:35:31 2022] DELETE /identity/v3/users/6ab36e845b514205b6a652827dab9d76 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:35:32.168728 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ee6b65-895b-4a13-87de-e937f0a5b29e None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:35:32.169006 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ee6b65-895b-4a13-87de-e937f0a5b29e None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:35:32.169278 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ee6b65-895b-4a13-87de-e937f0a5b29e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:35:32.188349 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:35:32.194135 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-65ee6b65-895b-4a13-87de-e937f0a5b29e None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:35:32.195520 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-65ee6b65-895b-4a13-87de-e937f0a5b29e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:35:32.276615 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 948/1891] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:35:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:35:32.288774 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1495fd13-f2b6-48b8-862e-6831253b462a admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:35:32.290508 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-1495fd13-f2b6-48b8-862e-6831253b462a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:35:32.373934 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1495fd13-f2b6-48b8-862e-6831253b462a admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:35:32.379111 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44fa881-dc4e-41c0-b9ca-69adafd5b440 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:35:32.380037 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44fa881-dc4e-41c0-b9ca-69adafd5b440 service neutron] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:35:32.380319 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44fa881-dc4e-41c0-b9ca-69adafd5b440 service neutron] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:35:32.380590 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44fa881-dc4e-41c0-b9ca-69adafd5b440 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:35:32.397173 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44fa881-dc4e-41c0-b9ca-69adafd5b440 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:35:32.398683 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44fa881-dc4e-41c0-b9ca-69adafd5b440 service neutron] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:35:32.419608 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 944/1892] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:35:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:35:32.672481 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af3b2f7-f4f6-469d-8918-813c1f425958 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:35:32.672766 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af3b2f7-f4f6-469d-8918-813c1f425958 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:35:32.673038 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af3b2f7-f4f6-469d-8918-813c1f425958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:35:32.692656 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-0af3b2f7-f4f6-469d-8918-813c1f425958 None None] MFA Rules not processed for user `db2e978bcb824a5e9d593b861c9e3bd7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:35:32.693842 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-0af3b2f7-f4f6-469d-8918-813c1f425958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:35:32.780822 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 949/1893] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:35:32 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:35:32.897618 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44fa881-dc4e-41c0-b9ca-69adafd5b440 service neutron] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:35:32.899276 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-e44fa881-dc4e-41c0-b9ca-69adafd5b440 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:35:32.914344 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44fa881-dc4e-41c0-b9ca-69adafd5b440 service neutron] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:35:32.945011 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7460370-7e6b-4af6-98dd-454271ea5ad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:35:32.945684 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7460370-7e6b-4af6-98dd-454271ea5ad8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:35:32.945971 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7460370-7e6b-4af6-98dd-454271ea5ad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:35:32.946248 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7460370-7e6b-4af6-98dd-454271ea5ad8 admin admin] PATH_INFO: `/v3/projects/f07d0f58ce774f03b519526b18a69d37` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:35:32.948160 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7460370-7e6b-4af6-98dd-454271ea5ad8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f07d0f58ce774f03b519526b18a69d37)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:35:32.952495 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7460370-7e6b-4af6-98dd-454271ea5ad8 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:35:33.041266 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-d7460370-7e6b-4af6-98dd-454271ea5ad8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f07d0f58ce774f03b519526b18a69d37', 'request_id': 'req-d7460370-7e6b-4af6-98dd-454271ea5ad8'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:35:33.049974 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-d7460370-7e6b-4af6-98dd-454271ea5ad8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f07d0f58ce774f03b519526b18a69d37', 'request_id': 'req-d7460370-7e6b-4af6-98dd-454271ea5ad8'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:35:33.052988 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:35:33.068122 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-d7460370-7e6b-4af6-98dd-454271ea5ad8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f07d0f58ce774f03b519526b18a69d37', 'request_id': 'req-d7460370-7e6b-4af6-98dd-454271ea5ad8'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:35:33.090879 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-d7460370-7e6b-4af6-98dd-454271ea5ad8 admin admin] The token cache is being invalidate because project f07d0f58ce774f03b519526b18a69d37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:35:33.091213 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-d7460370-7e6b-4af6-98dd-454271ea5ad8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:35:33.091871 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-d7460370-7e6b-4af6-98dd-454271ea5ad8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:35:33.093384 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 945/1894] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:35:32 2022] DELETE /identity/v3/projects/f07d0f58ce774f03b519526b18a69d37 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:35:33.098968 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b22c7e-30ed-4c5d-a472-dea92d93ef50 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:35:33.099556 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b22c7e-30ed-4c5d-a472-dea92d93ef50 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:35:33.100599 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b22c7e-30ed-4c5d-a472-dea92d93ef50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:35:33.135003 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-66b22c7e-30ed-4c5d-a472-dea92d93ef50 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:35:33.136986 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-66b22c7e-30ed-4c5d-a472-dea92d93ef50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:35:33.229452 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 950/1895] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:35:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:35:33.239815 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f6df5f-7d44-4de5-a53c-289187ae9804 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:35:33.240223 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f6df5f-7d44-4de5-a53c-289187ae9804 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:35:33.240738 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f6df5f-7d44-4de5-a53c-289187ae9804 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:35:33.280777 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-25f6df5f-7d44-4de5-a53c-289187ae9804 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:35:33.282411 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-25f6df5f-7d44-4de5-a53c-289187ae9804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:35:33.294694 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 946/1896] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:35:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:35:33.299063 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b22c7e-30ed-4c5d-a472-dea92d93ef50 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:35:33.312719 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b22c7e-30ed-4c5d-a472-dea92d93ef50 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:35:33.318233 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eff70a-8da6-45ba-9b49-1515c024eaed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:35:33.318849 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eff70a-8da6-45ba-9b49-1515c024eaed admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:35:33.319128 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eff70a-8da6-45ba-9b49-1515c024eaed admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:35:33.319449 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eff70a-8da6-45ba-9b49-1515c024eaed admin admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:35:33.320096 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82eff70a-8da6-45ba-9b49-1515c024eaed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:35:33.320524 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82eff70a-8da6-45ba-9b49-1515c024eaed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:35:33.322149 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82eff70a-8da6-45ba-9b49-1515c024eaed admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:35:33.339665 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 951/1897] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Oct 31 12:35:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:35:33.343502 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f6df5f-7d44-4de5-a53c-289187ae9804 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:35:33.356586 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f6df5f-7d44-4de5-a53c-289187ae9804 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:35:33.362214 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60388dd9-6778-4e95-a31f-b6e41fad14dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:35:33.363434 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60388dd9-6778-4e95-a31f-b6e41fad14dd admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:35:33.363722 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60388dd9-6778-4e95-a31f-b6e41fad14dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:35:33.364331 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60388dd9-6778-4e95-a31f-b6e41fad14dd admin admin] PATH_INFO: `/v3/projects` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:35:33.364914 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60388dd9-6778-4e95-a31f-b6e41fad14dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:35:33.367248 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60388dd9-6778-4e95-a31f-b6e41fad14dd admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:35:33.399421 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 947/1898] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:35:33 2022] POST /identity/v3/projects => generated 397 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:35:33.403699 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eff70a-8da6-45ba-9b49-1515c024eaed admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:35:33.417762 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eff70a-8da6-45ba-9b49-1515c024eaed admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:35:33.465247 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:35:33.489464 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd92ee7-a441-426e-855f-8832108f95e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:35:33.490170 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd92ee7-a441-426e-855f-8832108f95e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:35:33.490486 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd92ee7-a441-426e-855f-8832108f95e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:35:33.490759 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd92ee7-a441-426e-855f-8832108f95e3 admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:35:33.491574 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd92ee7-a441-426e-855f-8832108f95e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:35:33.493527 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd92ee7-a441-426e-855f-8832108f95e3 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:35:33.523012 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 952/1899] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:35:33 2022] POST /identity/v3/users => generated 299 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 31 12:35:33.527982 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60388dd9-6778-4e95-a31f-b6e41fad14dd admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:35:33.547316 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60388dd9-6778-4e95-a31f-b6e41fad14dd admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:35:33.553150 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea910e6-0c42-47e8-9055-c7611dcd775d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:35:33.553761 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea910e6-0c42-47e8-9055-c7611dcd775d admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:35:33.554041 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea910e6-0c42-47e8-9055-c7611dcd775d admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:35:33.554696 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea910e6-0c42-47e8-9055-c7611dcd775d admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:35:33.554957 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea910e6-0c42-47e8-9055-c7611dcd775d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:35:33.556841 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea910e6-0c42-47e8-9055-c7611dcd775d admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:35:33.589345 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 948/1900] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:35:33 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:35:33.594001 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd92ee7-a441-426e-855f-8832108f95e3 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:35:33.608961 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd92ee7-a441-426e-855f-8832108f95e3 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:35:33.616080 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93155a87-8a6b-45a2-b0bb-6b54fdeb04a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:35:33.616736 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93155a87-8a6b-45a2-b0bb-6b54fdeb04a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:35:33.617100 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93155a87-8a6b-45a2-b0bb-6b54fdeb04a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:35:33.617440 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93155a87-8a6b-45a2-b0bb-6b54fdeb04a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:35:33.618270 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93155a87-8a6b-45a2-b0bb-6b54fdeb04a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:35:33.620220 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93155a87-8a6b-45a2-b0bb-6b54fdeb04a7 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:35:33.643204 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 953/1901] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:35:33 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 31 12:35:33.644645 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea910e6-0c42-47e8-9055-c7611dcd775d admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:35:33.656834 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea910e6-0c42-47e8-9055-c7611dcd775d admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:35:33.662022 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e90a90-2ff8-42b6-83ef-65ddee9a63c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:35:33.662682 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e90a90-2ff8-42b6-83ef-65ddee9a63c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:35:33.662960 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e90a90-2ff8-42b6-83ef-65ddee9a63c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:35:33.663273 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e90a90-2ff8-42b6-83ef-65ddee9a63c0 admin admin] PATH_INFO: `/v3/projects/737129601abb49419d9ce27953650919/users/0143a6297e234426b9280efe699e0c46/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:35:33.681135 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e90a90-2ff8-42b6-83ef-65ddee9a63c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=737129601abb49419d9ce27953650919, user_id=0143a6297e234426b9280efe699e0c46, role_id=ca1a5695f8bd499aa0b8cedaeecd2cc6)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:35:33.684148 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e90a90-2ff8-42b6-83ef-65ddee9a63c0 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:35:33.699206 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 949/1902] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:35:33 2022] PUT /identity/v3/projects/737129601abb49419d9ce27953650919/users/0143a6297e234426b9280efe699e0c46/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 31 12:35:33.710599 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cec681-3f6a-4331-acb2-a6fbe24ed173 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:35:33.710599 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cec681-3f6a-4331-acb2-a6fbe24ed173 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:35:33.710869 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cec681-3f6a-4331-acb2-a6fbe24ed173 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:35:33.751056 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-62cec681-3f6a-4331-acb2-a6fbe24ed173 None None] MFA Rules not processed for user `0143a6297e234426b9280efe699e0c46`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:35:33.759617 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-62cec681-3f6a-4331-acb2-a6fbe24ed173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:35:33.863312 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 954/1903] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:35:33 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:35:33.874407 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e90a90-2ff8-42b6-83ef-65ddee9a63c0 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:35:33.878062 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-41e90a90-2ff8-42b6-83ef-65ddee9a63c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:35:33.950856 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e90a90-2ff8-42b6-83ef-65ddee9a63c0 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:35:33.955193 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1663b504-f096-4a4b-b82b-7931c8123096 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d6d7ea077d49e8a66c6c6762499702', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:35:33.955801 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1663b504-f096-4a4b-b82b-7931c8123096 service cinder] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:35:33.956192 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1663b504-f096-4a4b-b82b-7931c8123096 service cinder] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:35:33.956464 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1663b504-f096-4a4b-b82b-7931c8123096 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:35:33.968676 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1663b504-f096-4a4b-b82b-7931c8123096 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:35:33.970866 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1663b504-f096-4a4b-b82b-7931c8123096 service cinder] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:35:33.992359 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 950/1904] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:35:33 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:35:34.693455 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cec681-3f6a-4331-acb2-a6fbe24ed173 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:35:34.695039 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-62cec681-3f6a-4331-acb2-a6fbe24ed173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:35:34.760336 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cec681-3f6a-4331-acb2-a6fbe24ed173 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:35:34.765108 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed989757-b68a-4e32-a4f1-0eb2aec9fac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:35:34.765744 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed989757-b68a-4e32-a4f1-0eb2aec9fac4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:35:34.766016 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed989757-b68a-4e32-a4f1-0eb2aec9fac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:35:34.766283 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed989757-b68a-4e32-a4f1-0eb2aec9fac4 admin admin] PATH_INFO: `/v3/users/356fce5f98ab43be8bbbca23fbbaa02d` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:35:34.767504 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed989757-b68a-4e32-a4f1-0eb2aec9fac4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=356fce5f98ab43be8bbbca23fbbaa02d)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:35:34.769364 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed989757-b68a-4e32-a4f1-0eb2aec9fac4 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:35:34.826277 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-ed989757-b68a-4e32-a4f1-0eb2aec9fac4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '356fce5f98ab43be8bbbca23fbbaa02d'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:35:34.829721 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:35:34.839074 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-ed989757-b68a-4e32-a4f1-0eb2aec9fac4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '356fce5f98ab43be8bbbca23fbbaa02d'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:35:34.840067 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-ed989757-b68a-4e32-a4f1-0eb2aec9fac4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '356fce5f98ab43be8bbbca23fbbaa02d'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:35:34.841019 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-ed989757-b68a-4e32-a4f1-0eb2aec9fac4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '356fce5f98ab43be8bbbca23fbbaa02d'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:35:34.862618 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-ed989757-b68a-4e32-a4f1-0eb2aec9fac4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '356fce5f98ab43be8bbbca23fbbaa02d'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:35:34.882427 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 955/1905] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:35:34 2022] DELETE /identity/v3/users/356fce5f98ab43be8bbbca23fbbaa02d => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:35:34.887793 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d1df75-b82b-41e2-b1e9-f83421c2aea2 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:35:34.888397 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d1df75-b82b-41e2-b1e9-f83421c2aea2 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:35:34.888890 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d1df75-b82b-41e2-b1e9-f83421c2aea2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:35:34.911771 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-08d1df75-b82b-41e2-b1e9-f83421c2aea2 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:35:34.913068 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-08d1df75-b82b-41e2-b1e9-f83421c2aea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:35:34.994630 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 951/1906] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:35:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 31 12:35:35.005015 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed989757-b68a-4e32-a4f1-0eb2aec9fac4 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:35:35.006431 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-ed989757-b68a-4e32-a4f1-0eb2aec9fac4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:35:35.076246 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed989757-b68a-4e32-a4f1-0eb2aec9fac4 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:35:35.080261 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4616655a-a1e5-400d-83b9-d7cb4dc56558 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:35:35.080893 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4616655a-a1e5-400d-83b9-d7cb4dc56558 service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:35:35.081183 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4616655a-a1e5-400d-83b9-d7cb4dc56558 service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:35:35.081445 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4616655a-a1e5-400d-83b9-d7cb4dc56558 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:35:35.091392 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4616655a-a1e5-400d-83b9-d7cb4dc56558 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:35:35.092833 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4616655a-a1e5-400d-83b9-d7cb4dc56558 service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:35:35.103701 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 956/1907] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:35:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:35:35.793239 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d1df75-b82b-41e2-b1e9-f83421c2aea2 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:35:35.794795 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-08d1df75-b82b-41e2-b1e9-f83421c2aea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:35:35.807746 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d1df75-b82b-41e2-b1e9-f83421c2aea2 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:35:35.812363 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9efcc73-0406-4d46-8f2f-26a0599d8506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:35:35.812997 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9efcc73-0406-4d46-8f2f-26a0599d8506 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:35:35.813319 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9efcc73-0406-4d46-8f2f-26a0599d8506 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:35:35.813594 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9efcc73-0406-4d46-8f2f-26a0599d8506 admin admin] PATH_INFO: `/v3/projects/1985e64225234725930ce72abeac9ed4` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:35:35.815344 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9efcc73-0406-4d46-8f2f-26a0599d8506 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1985e64225234725930ce72abeac9ed4)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:35:35.817309 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9efcc73-0406-4d46-8f2f-26a0599d8506 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:35:35.860119 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:INSERT=3,keystone:DELETE=1,keystone:SELECT=96 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:35:35.888197 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-e9efcc73-0406-4d46-8f2f-26a0599d8506 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1985e64225234725930ce72abeac9ed4', 'request_id': 'req-e9efcc73-0406-4d46-8f2f-26a0599d8506'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:35:35.894477 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-e9efcc73-0406-4d46-8f2f-26a0599d8506 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1985e64225234725930ce72abeac9ed4', 'request_id': 'req-e9efcc73-0406-4d46-8f2f-26a0599d8506'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:35:35.901688 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-e9efcc73-0406-4d46-8f2f-26a0599d8506 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1985e64225234725930ce72abeac9ed4', 'request_id': 'req-e9efcc73-0406-4d46-8f2f-26a0599d8506'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:35:35.913909 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-e9efcc73-0406-4d46-8f2f-26a0599d8506 admin admin] The token cache is being invalidate because project 1985e64225234725930ce72abeac9ed4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:35:35.914196 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-e9efcc73-0406-4d46-8f2f-26a0599d8506 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:35:35.914837 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-e9efcc73-0406-4d46-8f2f-26a0599d8506 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:35:35.916566 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 952/1908] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:35:35 2022] DELETE /identity/v3/projects/1985e64225234725930ce72abeac9ed4 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:35:42.228835 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4616655a-a1e5-400d-83b9-d7cb4dc56558 service neutron] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:35:42.230303 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-4616655a-a1e5-400d-83b9-d7cb4dc56558 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:35:42.243087 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4616655a-a1e5-400d-83b9-d7cb4dc56558 service neutron] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:35:42.299845 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df8690e-2909-4cbb-8a35-045a7b6cef62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:35:42.300502 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df8690e-2909-4cbb-8a35-045a7b6cef62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:35:42.300777 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df8690e-2909-4cbb-8a35-045a7b6cef62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:35:42.301045 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df8690e-2909-4cbb-8a35-045a7b6cef62 admin admin] PATH_INFO: `/v3/users/0143a6297e234426b9280efe699e0c46` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:35:42.302251 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df8690e-2909-4cbb-8a35-045a7b6cef62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0143a6297e234426b9280efe699e0c46)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:35:42.304266 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df8690e-2909-4cbb-8a35-045a7b6cef62 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:35:42.349857 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-3df8690e-2909-4cbb-8a35-045a7b6cef62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0143a6297e234426b9280efe699e0c46'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:35:42.357318 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-3df8690e-2909-4cbb-8a35-045a7b6cef62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0143a6297e234426b9280efe699e0c46'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:35:42.357972 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-3df8690e-2909-4cbb-8a35-045a7b6cef62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0143a6297e234426b9280efe699e0c46'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:35:42.358564 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-3df8690e-2909-4cbb-8a35-045a7b6cef62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0143a6297e234426b9280efe699e0c46'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:35:42.371231 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-3df8690e-2909-4cbb-8a35-045a7b6cef62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0143a6297e234426b9280efe699e0c46'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:35:42.372393 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:35:42.385797 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 957/1909] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:35:42 2022] DELETE /identity/v3/users/0143a6297e234426b9280efe699e0c46 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:35:42.390255 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341ca167-1b36-4a84-8923-fdbe3d2296c9 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:35:42.390742 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341ca167-1b36-4a84-8923-fdbe3d2296c9 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:35:42.390839 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341ca167-1b36-4a84-8923-fdbe3d2296c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:35:42.410430 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-341ca167-1b36-4a84-8923-fdbe3d2296c9 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:35:42.411675 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-341ca167-1b36-4a84-8923-fdbe3d2296c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:35:42.469216 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 953/1910] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:35:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:35:42.478980 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df8690e-2909-4cbb-8a35-045a7b6cef62 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:35:42.480394 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-3df8690e-2909-4cbb-8a35-045a7b6cef62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:35:42.537851 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df8690e-2909-4cbb-8a35-045a7b6cef62 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:35:42.541488 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1c9fec-601a-4a5c-bb7e-527a6b7cb8ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:35:42.542087 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1c9fec-601a-4a5c-bb7e-527a6b7cb8ef service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:35:42.542362 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1c9fec-601a-4a5c-bb7e-527a6b7cb8ef service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:35:42.542631 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1c9fec-601a-4a5c-bb7e-527a6b7cb8ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:35:42.552287 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1c9fec-601a-4a5c-bb7e-527a6b7cb8ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:35:42.553631 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1c9fec-601a-4a5c-bb7e-527a6b7cb8ef service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:35:42.562836 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 958/1911] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:35:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:35:42.929215 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341ca167-1b36-4a84-8923-fdbe3d2296c9 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:35:42.930790 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-341ca167-1b36-4a84-8923-fdbe3d2296c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:35:42.943366 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341ca167-1b36-4a84-8923-fdbe3d2296c9 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:35:42.948049 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea78327c-b68a-4e0b-9fd2-fe1713155995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:35:42.948689 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea78327c-b68a-4e0b-9fd2-fe1713155995 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:35:42.948970 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea78327c-b68a-4e0b-9fd2-fe1713155995 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:35:42.949318 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea78327c-b68a-4e0b-9fd2-fe1713155995 admin admin] PATH_INFO: `/v3/projects/737129601abb49419d9ce27953650919` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:35:42.950490 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea78327c-b68a-4e0b-9fd2-fe1713155995 admin admin] RBAC: Authorizing `identity:delete_project(project_id=737129601abb49419d9ce27953650919)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:35:42.952433 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea78327c-b68a-4e0b-9fd2-fe1713155995 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:35:43.012122 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-ea78327c-b68a-4e0b-9fd2-fe1713155995 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '737129601abb49419d9ce27953650919', 'request_id': 'req-ea78327c-b68a-4e0b-9fd2-fe1713155995'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:35:43.018459 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-ea78327c-b68a-4e0b-9fd2-fe1713155995 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '737129601abb49419d9ce27953650919', 'request_id': 'req-ea78327c-b68a-4e0b-9fd2-fe1713155995'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:35:43.034520 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-ea78327c-b68a-4e0b-9fd2-fe1713155995 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '737129601abb49419d9ce27953650919', 'request_id': 'req-ea78327c-b68a-4e0b-9fd2-fe1713155995'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:35:43.048499 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-ea78327c-b68a-4e0b-9fd2-fe1713155995 admin admin] The token cache is being invalidate because project 737129601abb49419d9ce27953650919 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:35:43.049401 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-ea78327c-b68a-4e0b-9fd2-fe1713155995 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:35:43.050083 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-ea78327c-b68a-4e0b-9fd2-fe1713155995 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:35:43.051675 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 954/1912] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:35:42 2022] DELETE /identity/v3/projects/737129601abb49419d9ce27953650919 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:35:43.056771 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b3ef4a-eb63-4970-8996-043858435fa1 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:35:43.057067 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b3ef4a-eb63-4970-8996-043858435fa1 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:35:43.057408 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b3ef4a-eb63-4970-8996-043858435fa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:35:43.078901 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-58b3ef4a-eb63-4970-8996-043858435fa1 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:35:43.080506 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-58b3ef4a-eb63-4970-8996-043858435fa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:35:43.160206 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 959/1913] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:35:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 31 12:35:43.169424 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74ee0f6-7aa4-4fcf-916a-71f1df636cdd None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:35:43.169720 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74ee0f6-7aa4-4fcf-916a-71f1df636cdd None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:35:43.169973 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74ee0f6-7aa4-4fcf-916a-71f1df636cdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:35:43.199057 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-f74ee0f6-7aa4-4fcf-916a-71f1df636cdd None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:35:43.200524 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-f74ee0f6-7aa4-4fcf-916a-71f1df636cdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:35:43.209825 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 955/1914] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:35:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:35:43.214059 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b3ef4a-eb63-4970-8996-043858435fa1 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:35:43.224609 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b3ef4a-eb63-4970-8996-043858435fa1 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:35:43.229876 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ad8a0a-1248-4d6e-bd4d-f4a7135bfc17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:35:43.230489 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ad8a0a-1248-4d6e-bd4d-f4a7135bfc17 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:35:43.230755 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ad8a0a-1248-4d6e-bd4d-f4a7135bfc17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:35:43.231027 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ad8a0a-1248-4d6e-bd4d-f4a7135bfc17 admin admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:35:43.232018 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ad8a0a-1248-4d6e-bd4d-f4a7135bfc17 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:35:43.232018 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ad8a0a-1248-4d6e-bd4d-f4a7135bfc17 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:35:43.233572 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ad8a0a-1248-4d6e-bd4d-f4a7135bfc17 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:35:43.252805 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 960/1915] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Oct 31 12:35:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:35:43.253917 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74ee0f6-7aa4-4fcf-916a-71f1df636cdd None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:35:43.263663 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74ee0f6-7aa4-4fcf-916a-71f1df636cdd None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:35:43.269128 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7362ffd4-86ba-42a3-82b7-b48837959c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:35:43.269371 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7362ffd4-86ba-42a3-82b7-b48837959c2b admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:35:43.269613 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7362ffd4-86ba-42a3-82b7-b48837959c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:35:43.269886 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7362ffd4-86ba-42a3-82b7-b48837959c2b admin admin] PATH_INFO: `/v3/projects` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:35:43.270888 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7362ffd4-86ba-42a3-82b7-b48837959c2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:35:43.272950 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7362ffd4-86ba-42a3-82b7-b48837959c2b admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:35:43.308461 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 956/1916] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:35:43 2022] POST /identity/v3/projects => generated 379 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:35:43.310060 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ad8a0a-1248-4d6e-bd4d-f4a7135bfc17 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:35:43.321989 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ad8a0a-1248-4d6e-bd4d-f4a7135bfc17 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:35:43.384761 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f57a8c7-a9b9-48a1-b0d3-d49d3b337514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:35:43.385490 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f57a8c7-a9b9-48a1-b0d3-d49d3b337514 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:35:43.385778 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f57a8c7-a9b9-48a1-b0d3-d49d3b337514 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:35:43.386053 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f57a8c7-a9b9-48a1-b0d3-d49d3b337514 admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:35:43.386686 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f57a8c7-a9b9-48a1-b0d3-d49d3b337514 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:35:43.388672 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f57a8c7-a9b9-48a1-b0d3-d49d3b337514 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:35:43.412083 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 961/1917] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:35:43 2022] POST /identity/v3/users => generated 290 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:35:43.417291 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7362ffd4-86ba-42a3-82b7-b48837959c2b admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:35:43.425312 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:35:43.430347 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7362ffd4-86ba-42a3-82b7-b48837959c2b admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:35:43.437957 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe8f214-9979-4bb6-adab-dbb002934d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:35:43.440894 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe8f214-9979-4bb6-adab-dbb002934d65 admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:35:43.441454 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe8f214-9979-4bb6-adab-dbb002934d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:35:43.441977 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe8f214-9979-4bb6-adab-dbb002934d65 admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:35:43.442827 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe8f214-9979-4bb6-adab-dbb002934d65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:35:43.445568 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe8f214-9979-4bb6-adab-dbb002934d65 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:35:43.465453 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f57a8c7-a9b9-48a1-b0d3-d49d3b337514 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:35:43.466544 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 957/1918] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:35:43 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:35:43.476388 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f57a8c7-a9b9-48a1-b0d3-d49d3b337514 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:35:43.485222 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0e40d0-aa28-4112-9623-e3205c78f341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:35:43.486515 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0e40d0-aa28-4112-9623-e3205c78f341 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:35:43.486998 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0e40d0-aa28-4112-9623-e3205c78f341 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:35:43.487604 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0e40d0-aa28-4112-9623-e3205c78f341 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:35:43.488443 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0e40d0-aa28-4112-9623-e3205c78f341 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:35:43.490198 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0e40d0-aa28-4112-9623-e3205c78f341 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:35:43.495935 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:35:43.504217 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 962/1919] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:35:43 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:35:43.508195 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe8f214-9979-4bb6-adab-dbb002934d65 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:35:43.518710 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe8f214-9979-4bb6-adab-dbb002934d65 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:35:43.524334 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905f6636-047a-458a-9817-2690d21cf52d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:35:43.524952 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905f6636-047a-458a-9817-2690d21cf52d admin admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:35:43.525552 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905f6636-047a-458a-9817-2690d21cf52d admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:35:43.525552 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905f6636-047a-458a-9817-2690d21cf52d admin admin] PATH_INFO: `/v3/projects/8ae8c589d0b84ce096451a10717fc601/users/49b76d81ee9446cf9014a2e695ef03e7/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:35:43.542484 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905f6636-047a-458a-9817-2690d21cf52d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ae8c589d0b84ce096451a10717fc601, user_id=49b76d81ee9446cf9014a2e695ef03e7, role_id=ca1a5695f8bd499aa0b8cedaeecd2cc6)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:35:43.545156 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905f6636-047a-458a-9817-2690d21cf52d admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:35:43.564966 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 958/1920] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:35:43 2022] PUT /identity/v3/projects/8ae8c589d0b84ce096451a10717fc601/users/49b76d81ee9446cf9014a2e695ef03e7/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:35:43.567985 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67213e98-e444-4a69-95b0-5cd275c8efb6 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:35:43.567985 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67213e98-e444-4a69-95b0-5cd275c8efb6 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:35:43.567985 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67213e98-e444-4a69-95b0-5cd275c8efb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:35:43.589575 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-67213e98-e444-4a69-95b0-5cd275c8efb6 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:35:43.590792 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-67213e98-e444-4a69-95b0-5cd275c8efb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:35:43.657635 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 963/1921] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:35:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:35:43.667279 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905f6636-047a-458a-9817-2690d21cf52d admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:35:43.669004 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-905f6636-047a-458a-9817-2690d21cf52d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:35:43.727250 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905f6636-047a-458a-9817-2690d21cf52d admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:35:43.730901 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3e9c45-e646-45c7-a794-e0bcbaace655 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:35:43.731540 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3e9c45-e646-45c7-a794-e0bcbaace655 service neutron] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:35:43.731826 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3e9c45-e646-45c7-a794-e0bcbaace655 service neutron] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:35:43.732096 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3e9c45-e646-45c7-a794-e0bcbaace655 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:35:43.741223 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3e9c45-e646-45c7-a794-e0bcbaace655 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:35:43.742638 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3e9c45-e646-45c7-a794-e0bcbaace655 service neutron] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:35:43.752493 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 959/1922] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:35:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:35:46.573688 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2718f3f-7bfd-4263-93da-cce221d2913f None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:35:46.574219 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2718f3f-7bfd-4263-93da-cce221d2913f None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:35:46.574600 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2718f3f-7bfd-4263-93da-cce221d2913f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:35:46.596729 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-c2718f3f-7bfd-4263-93da-cce221d2913f None None] MFA Rules not processed for user `bf4faacf7df248d28f32d7014dadcc07`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:35:46.597947 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-c2718f3f-7bfd-4263-93da-cce221d2913f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:35:46.713260 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 964/1923] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Oct 31 12:35:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:35:46.727598 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3e9c45-e646-45c7-a794-e0bcbaace655 service neutron] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:35:46.729484 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-ba3e9c45-e646-45c7-a794-e0bcbaace655 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:35:46.810122 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3e9c45-e646-45c7-a794-e0bcbaace655 service neutron] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:35:46.817716 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f3b35a-b57f-4f93-a139-888eda8cbd08 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2254a5786c249b4aa79244f7198ae65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:35:46.819098 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f3b35a-b57f-4f93-a139-888eda8cbd08 service swift] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:35:46.819278 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f3b35a-b57f-4f93-a139-888eda8cbd08 service swift] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:35:46.819581 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f3b35a-b57f-4f93-a139-888eda8cbd08 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:35:46.828614 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f3b35a-b57f-4f93-a139-888eda8cbd08 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:35:46.830362 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f3b35a-b57f-4f93-a139-888eda8cbd08 service swift] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:35:46.839762 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 960/1924] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Oct 31 12:35:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 31 12:35:49.595584 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2684563-0922-4d65-813b-ae58239fcd66 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:35:49.595584 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2684563-0922-4d65-813b-ae58239fcd66 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:35:49.595584 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2684563-0922-4d65-813b-ae58239fcd66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:35:49.642429 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-b2684563-0922-4d65-813b-ae58239fcd66 None None] MFA Rules not processed for user `49b76d81ee9446cf9014a2e695ef03e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:35:49.646192 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-b2684563-0922-4d65-813b-ae58239fcd66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:35:49.793256 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 965/1925] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:35:49 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:35:49.794909 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f3b35a-b57f-4f93-a139-888eda8cbd08 service swift] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:35:49.814136 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f3b35a-b57f-4f93-a139-888eda8cbd08 service swift] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:35:49.825825 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03c79f5-e23b-48e9-a79a-4188314a5b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:35:49.826727 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03c79f5-e23b-48e9-a79a-4188314a5b6d admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:35:49.827225 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03c79f5-e23b-48e9-a79a-4188314a5b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:35:49.827712 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03c79f5-e23b-48e9-a79a-4188314a5b6d admin admin] PATH_INFO: `/v3/projects` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:35:49.828580 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03c79f5-e23b-48e9-a79a-4188314a5b6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:35:49.835585 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03c79f5-e23b-48e9-a79a-4188314a5b6d admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:35:49.889434 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 961/1926] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:35:49 2022] POST /identity/v3/projects => generated 377 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:35:49.890711 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2684563-0922-4d65-813b-ae58239fcd66 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:35:49.905944 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2684563-0922-4d65-813b-ae58239fcd66 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:35:50.013011 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:35:50.116547 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987a4bf0-4003-4cdf-8356-f7b874ac210b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:35:50.119344 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987a4bf0-4003-4cdf-8356-f7b874ac210b admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:35:50.119635 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987a4bf0-4003-4cdf-8356-f7b874ac210b admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:35:50.119910 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987a4bf0-4003-4cdf-8356-f7b874ac210b admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:35:50.120593 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987a4bf0-4003-4cdf-8356-f7b874ac210b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:35:50.124935 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987a4bf0-4003-4cdf-8356-f7b874ac210b admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:35:50.162532 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 966/1927] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:35:49 2022] POST /identity/v3/users => generated 295 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:35:50.166798 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03c79f5-e23b-48e9-a79a-4188314a5b6d admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:35:50.188749 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03c79f5-e23b-48e9-a79a-4188314a5b6d admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:35:50.209193 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264d5d59-162a-4ffa-bc07-b0231329c298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:35:50.209193 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264d5d59-162a-4ffa-bc07-b0231329c298 admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:35:50.209193 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264d5d59-162a-4ffa-bc07-b0231329c298 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:35:50.209484 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264d5d59-162a-4ffa-bc07-b0231329c298 admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:35:50.210509 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264d5d59-162a-4ffa-bc07-b0231329c298 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:35:50.212542 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264d5d59-162a-4ffa-bc07-b0231329c298 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:35:50.244247 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 962/1928] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:35:50 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:35:50.245338 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987a4bf0-4003-4cdf-8356-f7b874ac210b admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:35:50.259669 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987a4bf0-4003-4cdf-8356-f7b874ac210b admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:35:50.270876 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f1d647-12ac-4c4c-9fa5-3e0a04d3b7b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:35:50.271628 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f1d647-12ac-4c4c-9fa5-3e0a04d3b7b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:35:50.272894 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f1d647-12ac-4c4c-9fa5-3e0a04d3b7b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:35:50.273245 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f1d647-12ac-4c4c-9fa5-3e0a04d3b7b1 admin admin] PATH_INFO: `/v3/projects/164b302bf1bc417a9fd2affa94c91d40/users/7e800e81836347afb340983d74b26b1c/roles/176c07b4fe3b4537bdfeb4be21b379c5` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:35:50.321586 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f1d647-12ac-4c4c-9fa5-3e0a04d3b7b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=164b302bf1bc417a9fd2affa94c91d40, user_id=7e800e81836347afb340983d74b26b1c, role_id=176c07b4fe3b4537bdfeb4be21b379c5)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:35:50.324547 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f1d647-12ac-4c4c-9fa5-3e0a04d3b7b1 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:35:50.354825 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 967/1929] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:35:50 2022] PUT /identity/v3/projects/164b302bf1bc417a9fd2affa94c91d40/users/7e800e81836347afb340983d74b26b1c/roles/176c07b4fe3b4537bdfeb4be21b379c5 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:35:50.360096 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264d5d59-162a-4ffa-bc07-b0231329c298 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:35:50.377881 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264d5d59-162a-4ffa-bc07-b0231329c298 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:35:50.438152 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:35:50.496760 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e67faa-3849-44c5-a69e-4af57096066d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:35:50.497758 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e67faa-3849-44c5-a69e-4af57096066d admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:35:50.498303 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e67faa-3849-44c5-a69e-4af57096066d admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:35:50.498755 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e67faa-3849-44c5-a69e-4af57096066d admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:35:50.499606 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e67faa-3849-44c5-a69e-4af57096066d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:35:50.503045 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e67faa-3849-44c5-a69e-4af57096066d admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:35:50.517269 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 963/1930] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:35:50 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:35:50.521473 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f1d647-12ac-4c4c-9fa5-3e0a04d3b7b1 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:35:50.537015 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f1d647-12ac-4c4c-9fa5-3e0a04d3b7b1 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:35:50.544760 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9981e7-f2b8-4613-ac62-22085f32614d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:35:50.546519 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9981e7-f2b8-4613-ac62-22085f32614d admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:35:50.547064 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9981e7-f2b8-4613-ac62-22085f32614d admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:35:50.547564 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9981e7-f2b8-4613-ac62-22085f32614d admin admin] PATH_INFO: `/v3/projects/164b302bf1bc417a9fd2affa94c91d40/users/7e800e81836347afb340983d74b26b1c/roles/176c07b4fe3b4537bdfeb4be21b379c5` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:35:50.553641 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9981e7-f2b8-4613-ac62-22085f32614d admin admin] RBAC: Authorizing `identity:create_grant(project_id=164b302bf1bc417a9fd2affa94c91d40, user_id=7e800e81836347afb340983d74b26b1c, role_id=176c07b4fe3b4537bdfeb4be21b379c5)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:35:50.558534 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9981e7-f2b8-4613-ac62-22085f32614d admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:35:50.575666 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 968/1931] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:35:50 2022] PUT /identity/v3/projects/164b302bf1bc417a9fd2affa94c91d40/users/7e800e81836347afb340983d74b26b1c/roles/176c07b4fe3b4537bdfeb4be21b379c5 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:35:55.913056 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e67faa-3849-44c5-a69e-4af57096066d admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:35:55.914923 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-38e67faa-3849-44c5-a69e-4af57096066d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:35:56.077521 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e67faa-3849-44c5-a69e-4af57096066d admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:35:56.230566 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d603cd-67f6-4852-9c1c-2dc36b4d8e1e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e7777495d7a406c8c50b1ad52e47263', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:35:56.231657 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d603cd-67f6-4852-9c1c-2dc36b4d8e1e service placement] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:35:56.231657 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d603cd-67f6-4852-9c1c-2dc36b4d8e1e service placement] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:35:56.231869 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d603cd-67f6-4852-9c1c-2dc36b4d8e1e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:35:56.234763 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-74d603cd-67f6-4852-9c1c-2dc36b4d8e1e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:35:56.268770 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d603cd-67f6-4852-9c1c-2dc36b4d8e1e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:35:56.270693 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d603cd-67f6-4852-9c1c-2dc36b4d8e1e service placement] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:35:56.293838 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 964/1932] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:35:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:35:59.809500 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b01acd-251a-4941-8b81-d7324d1acd26 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:35:59.810333 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b01acd-251a-4941-8b81-d7324d1acd26 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:35:59.810827 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b01acd-251a-4941-8b81-d7324d1acd26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:35:59.866457 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-61b01acd-251a-4941-8b81-d7324d1acd26 None None] MFA Rules not processed for user `7e800e81836347afb340983d74b26b1c`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:35:59.867659 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-61b01acd-251a-4941-8b81-d7324d1acd26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:36:00.019286 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 969/1933] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:35:59 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:36:00.036746 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d603cd-67f6-4852-9c1c-2dc36b4d8e1e service placement] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:36:00.038326 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-74d603cd-67f6-4852-9c1c-2dc36b4d8e1e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:36:00.121495 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:36:00.156003 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d603cd-67f6-4852-9c1c-2dc36b4d8e1e service placement] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:36:00.165232 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf128ce-9aae-4582-a2bd-e769d8618777 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99889b1dab0b471d9c77887cffefc652', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:36:00.165232 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf128ce-9aae-4582-a2bd-e769d8618777 service nova] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:36:00.165232 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf128ce-9aae-4582-a2bd-e769d8618777 service nova] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:36:00.165610 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf128ce-9aae-4582-a2bd-e769d8618777 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:36:00.233861 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf128ce-9aae-4582-a2bd-e769d8618777 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:36:00.238327 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf128ce-9aae-4582-a2bd-e769d8618777 service nova] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:36:00.248300 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 965/1934] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:36:00 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:36:08.676161 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c517c8-46db-4540-9b2a-0078af72dcd9 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:36:08.676730 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c517c8-46db-4540-9b2a-0078af72dcd9 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:36:08.676840 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c517c8-46db-4540-9b2a-0078af72dcd9 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:36:08.678149 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 970/1935] 10.222.0.15 () {54 vars in 921 bytes} [Mon Oct 31 12:36:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:36:08.686930 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc58509f-f0d8-49ae-a778-c6bbbcee2e0b None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:36:08.687512 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc58509f-f0d8-49ae-a778-c6bbbcee2e0b None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:36:08.687512 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc58509f-f0d8-49ae-a778-c6bbbcee2e0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:36:08.707910 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-fc58509f-f0d8-49ae-a778-c6bbbcee2e0b None None] MFA Rules not processed for user `99889b1dab0b471d9c77887cffefc652`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:36:08.709148 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-fc58509f-f0d8-49ae-a778-c6bbbcee2e0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:36:08.718362 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 966/1936] 10.222.0.15 () {58 vars in 1018 bytes} [Mon Oct 31 12:36:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:36:08.733985 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c517c8-46db-4540-9b2a-0078af72dcd9 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:36:08.735941 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-68c517c8-46db-4540-9b2a-0078af72dcd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:36:08.799086 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c517c8-46db-4540-9b2a-0078af72dcd9 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:36:08.802717 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e1eb1a-f148-4be1-9fb0-9c7d00731ccd service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4feaa01c5ed64b65b10d4dece823702c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:36:08.803361 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e1eb1a-f148-4be1-9fb0-9c7d00731ccd service glance] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:36:08.803641 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e1eb1a-f148-4be1-9fb0-9c7d00731ccd service glance] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:36:08.803913 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e1eb1a-f148-4be1-9fb0-9c7d00731ccd service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:36:08.819301 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e1eb1a-f148-4be1-9fb0-9c7d00731ccd service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:36:08.819301 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e1eb1a-f148-4be1-9fb0-9c7d00731ccd service glance] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:36:08.827653 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 971/1937] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:36:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:36:08.900442 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae968762-7f63-4e35-a3a4-3d4f857cceff None None] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:36:08.900766 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae968762-7f63-4e35-a3a4-3d4f857cceff None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:36:08.901149 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae968762-7f63-4e35-a3a4-3d4f857cceff None None] PATH_INFO: `/` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:36:08.902638 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 967/1938] 10.222.0.15 () {54 vars in 921 bytes} [Mon Oct 31 12:36:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:36:08.909269 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9421ffb5-44e8-423a-94e5-67ec8438cf42 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:36:08.909648 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9421ffb5-44e8-423a-94e5-67ec8438cf42 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:36:08.909931 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9421ffb5-44e8-423a-94e5-67ec8438cf42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:36:08.930002 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-9421ffb5-44e8-423a-94e5-67ec8438cf42 None None] MFA Rules not processed for user `25e9c7d1e4714bf18a1ced84855423d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:36:08.931231 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-9421ffb5-44e8-423a-94e5-67ec8438cf42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:36:08.939723 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:36:08.985515 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 972/1939] 10.222.0.15 () {58 vars in 1018 bytes} [Mon Oct 31 12:36:08 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 31 12:36:08.997341 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae968762-7f63-4e35-a3a4-3d4f857cceff None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:36:08.999049 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-ae968762-7f63-4e35-a3a4-3d4f857cceff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:36:09.012775 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae968762-7f63-4e35-a3a4-3d4f857cceff None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:36:09.016431 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb885a3-b22a-4662-9dfc-3595aea9bd0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:36:09.017040 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb885a3-b22a-4662-9dfc-3595aea9bd0d service neutron] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:36:09.017315 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb885a3-b22a-4662-9dfc-3595aea9bd0d service neutron] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:36:09.017588 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb885a3-b22a-4662-9dfc-3595aea9bd0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:36:09.026122 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb885a3-b22a-4662-9dfc-3595aea9bd0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:36:09.027621 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb885a3-b22a-4662-9dfc-3595aea9bd0d service neutron] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:36:09.036469 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 968/1940] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:36:08 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:36:09.122922 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470c0054-93a8-4b00-beda-2156f4b4dec8 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:36:09.123273 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470c0054-93a8-4b00-beda-2156f4b4dec8 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:36:09.123555 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470c0054-93a8-4b00-beda-2156f4b4dec8 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:36:09.124909 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 973/1941] 10.222.0.15 () {54 vars in 921 bytes} [Mon Oct 31 12:36:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:36:09.130529 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0324003-b62f-43f1-a42c-69ee0d1c2971 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:36:09.130809 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0324003-b62f-43f1-a42c-69ee0d1c2971 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:36:09.131080 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0324003-b62f-43f1-a42c-69ee0d1c2971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:36:09.151139 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-b0324003-b62f-43f1-a42c-69ee0d1c2971 None None] MFA Rules not processed for user `99889b1dab0b471d9c77887cffefc652`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:36:09.152470 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-b0324003-b62f-43f1-a42c-69ee0d1c2971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:36:09.163315 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 969/1942] 10.222.0.15 () {58 vars in 1018 bytes} [Mon Oct 31 12:36:09 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:36:09.174181 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470c0054-93a8-4b00-beda-2156f4b4dec8 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:36:09.185161 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470c0054-93a8-4b00-beda-2156f4b4dec8 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:36:09.188782 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9e550e-6a50-48e7-914a-61edb33848c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:36:09.189376 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9e550e-6a50-48e7-914a-61edb33848c7 service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:36:09.189657 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9e550e-6a50-48e7-914a-61edb33848c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:36:09.189927 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9e550e-6a50-48e7-914a-61edb33848c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:36:09.199863 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9e550e-6a50-48e7-914a-61edb33848c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:36:09.201260 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9e550e-6a50-48e7-914a-61edb33848c7 service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:36:09.212629 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 974/1943] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:36:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:36:12.173283 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e62597e-c439-4b97-8231-9068a0e57bbf None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:36:12.173832 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e62597e-c439-4b97-8231-9068a0e57bbf None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:36:12.173832 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e62597e-c439-4b97-8231-9068a0e57bbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:36:12.193396 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-7e62597e-c439-4b97-8231-9068a0e57bbf None None] MFA Rules not processed for user `bf4faacf7df248d28f32d7014dadcc07`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:36:12.194577 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-7e62597e-c439-4b97-8231-9068a0e57bbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:36:12.254132 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 970/1944] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Oct 31 12:36:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:36:12.267683 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9e550e-6a50-48e7-914a-61edb33848c7 service neutron] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:36:12.336621 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9e550e-6a50-48e7-914a-61edb33848c7 service neutron] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:36:12.341837 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a791f36e-ff98-4601-a0ac-bea104da7d8d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2254a5786c249b4aa79244f7198ae65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:36:12.342451 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a791f36e-ff98-4601-a0ac-bea104da7d8d service swift] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:36:12.343378 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a791f36e-ff98-4601-a0ac-bea104da7d8d service swift] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:36:12.343670 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a791f36e-ff98-4601-a0ac-bea104da7d8d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:36:12.352862 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a791f36e-ff98-4601-a0ac-bea104da7d8d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:36:12.354472 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a791f36e-ff98-4601-a0ac-bea104da7d8d service swift] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:36:12.364110 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 975/1945] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Oct 31 12:36:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 31 12:36:20.413032 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539e3ada-3a76-4202-8d04-3c9e45620940 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:36:20.413669 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539e3ada-3a76-4202-8d04-3c9e45620940 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:36:20.413669 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539e3ada-3a76-4202-8d04-3c9e45620940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:36:20.432810 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-539e3ada-3a76-4202-8d04-3c9e45620940 None None] MFA Rules not processed for user `49b76d81ee9446cf9014a2e695ef03e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:36:20.434164 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-539e3ada-3a76-4202-8d04-3c9e45620940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:36:20.441670 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 971/1946] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:36:20 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:36:22.359471 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=55 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:36:30.426985 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=70 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:36:36.515574 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec980f6-b7e6-426c-8a61-9133a7d954f2 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:36:36.516056 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec980f6-b7e6-426c-8a61-9133a7d954f2 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:36:36.516148 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec980f6-b7e6-426c-8a61-9133a7d954f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:36:36.536166 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-2ec980f6-b7e6-426c-8a61-9133a7d954f2 None None] MFA Rules not processed for user `bf4faacf7df248d28f32d7014dadcc07`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:36:36.537357 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-2ec980f6-b7e6-426c-8a61-9133a7d954f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:36:36.544458 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 976/1947] 10.222.0.15 () {58 vars in 967 bytes} [Mon Oct 31 12:36:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:36:36.556909 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539e3ada-3a76-4202-8d04-3c9e45620940 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:36:36.569022 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539e3ada-3a76-4202-8d04-3c9e45620940 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:36:36.573853 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a2977d-6ac9-4ce7-8ec0-da7f7792463b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2254a5786c249b4aa79244f7198ae65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:36:36.574479 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a2977d-6ac9-4ce7-8ec0-da7f7792463b service swift] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:36:36.575078 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a2977d-6ac9-4ce7-8ec0-da7f7792463b service swift] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:36:36.575078 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a2977d-6ac9-4ce7-8ec0-da7f7792463b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:36:36.584274 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a2977d-6ac9-4ce7-8ec0-da7f7792463b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:36:36.585828 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a2977d-6ac9-4ce7-8ec0-da7f7792463b service swift] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:36:36.595742 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 972/1948] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Oct 31 12:36:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 31 12:36:42.913451 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec980f6-b7e6-426c-8a61-9133a7d954f2 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:36:42.929921 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec980f6-b7e6-426c-8a61-9133a7d954f2 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:36:42.934211 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8520fb4d-cd2d-4ae0-a067-6b4b953d2338 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:36:42.936632 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8520fb4d-cd2d-4ae0-a067-6b4b953d2338 service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:36:42.937164 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8520fb4d-cd2d-4ae0-a067-6b4b953d2338 service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:36:42.937514 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8520fb4d-cd2d-4ae0-a067-6b4b953d2338 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:36:42.951537 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8520fb4d-cd2d-4ae0-a067-6b4b953d2338 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:36:42.953210 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8520fb4d-cd2d-4ae0-a067-6b4b953d2338 service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:36:42.971182 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 977/1949] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:36:42 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:36:46.592216 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=6 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:36:52.960710 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=9 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:36:58.451386 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fad017-348f-4ba2-bb67-277da8cc0b28 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:36:58.451884 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fad017-348f-4ba2-bb67-277da8cc0b28 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:36:58.451982 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fad017-348f-4ba2-bb67-277da8cc0b28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:36:58.473553 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-f2fad017-348f-4ba2-bb67-277da8cc0b28 None None] MFA Rules not processed for user `b2c03f0365ee4a26b8c076ae365325b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:36:58.474728 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-f2fad017-348f-4ba2-bb67-277da8cc0b28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:36:58.528632 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 973/1950] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:36:58 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:37:08.522031 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=25 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:37:10.270087 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1553059-6a32-456b-8078-e29ffd98aa95 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:37:10.270643 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1553059-6a32-456b-8078-e29ffd98aa95 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:37:10.270643 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1553059-6a32-456b-8078-e29ffd98aa95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:37:10.290737 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-b1553059-6a32-456b-8078-e29ffd98aa95 None None] MFA Rules not processed for user `db2e978bcb824a5e9d593b861c9e3bd7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:37:10.291967 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-b1553059-6a32-456b-8078-e29ffd98aa95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:37:10.350912 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 978/1951] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:37:10 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:37:20.344288 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=25 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:37:32.365910 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2035478b-e51e-4f64-bfd6-4c7782e1c779 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:37:32.366415 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2035478b-e51e-4f64-bfd6-4c7782e1c779 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:37:32.366518 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2035478b-e51e-4f64-bfd6-4c7782e1c779 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:37:32.384802 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-2035478b-e51e-4f64-bfd6-4c7782e1c779 None None] MFA Rules not processed for user `a6d1ace68aa84499a4929da60f3d71bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:37:32.386472 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-2035478b-e51e-4f64-bfd6-4c7782e1c779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:37:32.441094 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 974/1952] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:37:32 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:37:42.434044 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=25 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:37:45.683809 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1553059-6a32-456b-8078-e29ffd98aa95 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:37:45.695150 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1553059-6a32-456b-8078-e29ffd98aa95 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:37:45.700700 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56357d05-c313-4b91-85db-90fe4f2fd13d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:37:45.701329 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56357d05-c313-4b91-85db-90fe4f2fd13d service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:37:45.701802 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56357d05-c313-4b91-85db-90fe4f2fd13d service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:37:45.702778 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56357d05-c313-4b91-85db-90fe4f2fd13d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:37:45.705655 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-56357d05-c313-4b91-85db-90fe4f2fd13d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:37:45.723954 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56357d05-c313-4b91-85db-90fe4f2fd13d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:37:45.725494 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56357d05-c313-4b91-85db-90fe4f2fd13d service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:37:45.737633 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 979/1953] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:37:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:37:46.669328 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2035478b-e51e-4f64-bfd6-4c7782e1c779 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:37:46.680531 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2035478b-e51e-4f64-bfd6-4c7782e1c779 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:37:46.684377 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acc26f9-f742-4bcf-a7cb-5c7147a0af78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:37:46.685082 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc26f9-f742-4bcf-a7cb-5c7147a0af78 service neutron] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:37:46.685428 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc26f9-f742-4bcf-a7cb-5c7147a0af78 service neutron] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:37:46.685769 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc26f9-f742-4bcf-a7cb-5c7147a0af78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:37:46.687678 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-7acc26f9-f742-4bcf-a7cb-5c7147a0af78 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:37:46.702959 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acc26f9-f742-4bcf-a7cb-5c7147a0af78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:37:46.704619 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acc26f9-f742-4bcf-a7cb-5c7147a0af78 service neutron] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:37:46.719320 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 975/1954] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:37:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:37:50.061054 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7d70cc-31e6-4fbc-bea5-63de43756dc6 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:37:50.061583 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7d70cc-31e6-4fbc-bea5-63de43756dc6 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:37:50.061583 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7d70cc-31e6-4fbc-bea5-63de43756dc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:37:50.080694 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-9d7d70cc-31e6-4fbc-bea5-63de43756dc6 None None] MFA Rules not processed for user `d6c1beb321e44908ab987b17f7b74484`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:37:50.081923 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-9d7d70cc-31e6-4fbc-bea5-63de43756dc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:37:50.136510 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 980/1955] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:37:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:37:55.353808 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acc26f9-f742-4bcf-a7cb-5c7147a0af78 service neutron] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:37:55.364870 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acc26f9-f742-4bcf-a7cb-5c7147a0af78 service neutron] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:37:55.370364 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ed261e-9c0f-4b52-90f7-5583da9e2502 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99889b1dab0b471d9c77887cffefc652', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:37:55.370982 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ed261e-9c0f-4b52-90f7-5583da9e2502 service nova] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:37:55.371287 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ed261e-9c0f-4b52-90f7-5583da9e2502 service nova] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:37:55.371561 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ed261e-9c0f-4b52-90f7-5583da9e2502 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:37:55.373225 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-91ed261e-9c0f-4b52-90f7-5583da9e2502 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:37:55.383515 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ed261e-9c0f-4b52-90f7-5583da9e2502 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:37:55.385607 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ed261e-9c0f-4b52-90f7-5583da9e2502 service nova] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:37:55.394858 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 976/1956] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:37:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:37:57.086594 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7d70cc-31e6-4fbc-bea5-63de43756dc6 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:37:57.099520 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7d70cc-31e6-4fbc-bea5-63de43756dc6 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:37:57.105481 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8cd672-9f38-4938-a45c-fdb617d4c281 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99889b1dab0b471d9c77887cffefc652', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:37:57.106085 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8cd672-9f38-4938-a45c-fdb617d4c281 service nova] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:37:57.106364 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8cd672-9f38-4938-a45c-fdb617d4c281 service nova] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:37:57.106636 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8cd672-9f38-4938-a45c-fdb617d4c281 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:37:57.108372 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-3c8cd672-9f38-4938-a45c-fdb617d4c281 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:37:57.118404 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8cd672-9f38-4938-a45c-fdb617d4c281 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:37:57.120522 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8cd672-9f38-4938-a45c-fdb617d4c281 service nova] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:37:57.129351 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 981/1957] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:37:57 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:38:02.971855 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ed261e-9c0f-4b52-90f7-5583da9e2502 service nova] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:02.973304 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-91ed261e-9c0f-4b52-90f7-5583da9e2502 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:38:02.987956 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ed261e-9c0f-4b52-90f7-5583da9e2502 service nova] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:02.993468 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5e9253-473d-4390-84c8-095c348b0ba9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99889b1dab0b471d9c77887cffefc652', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:02.994119 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5e9253-473d-4390-84c8-095c348b0ba9 service nova] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:02.994394 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5e9253-473d-4390-84c8-095c348b0ba9 service nova] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:02.994666 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5e9253-473d-4390-84c8-095c348b0ba9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:02.996495 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-fa5e9253-473d-4390-84c8-095c348b0ba9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:38:03.007369 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5e9253-473d-4390-84c8-095c348b0ba9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:03.009413 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5e9253-473d-4390-84c8-095c348b0ba9 service nova] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:03.018611 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 977/1958] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:38:02 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:38:07.125191 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=37 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:38:13.014316 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=18 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:38:18.332181 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8cd672-9f38-4938-a45c-fdb617d4c281 service nova] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:18.344717 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8cd672-9f38-4938-a45c-fdb617d4c281 service nova] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:18.348641 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb398dff-b74d-43ac-a3c0-ed15696d7a35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:18.349255 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb398dff-b74d-43ac-a3c0-ed15696d7a35 service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:18.349537 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb398dff-b74d-43ac-a3c0-ed15696d7a35 service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:18.349843 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb398dff-b74d-43ac-a3c0-ed15696d7a35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:18.351857 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-bb398dff-b74d-43ac-a3c0-ed15696d7a35 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:38:18.424204 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb398dff-b74d-43ac-a3c0-ed15696d7a35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:18.425694 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb398dff-b74d-43ac-a3c0-ed15696d7a35 service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:18.435932 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 982/1959] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:38:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:38:22.084568 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c49f51-5e44-4d77-aed7-cf3178ba1447 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:22.085082 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c49f51-5e44-4d77-aed7-cf3178ba1447 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:22.085082 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c49f51-5e44-4d77-aed7-cf3178ba1447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:22.102303 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1f39cd-7994-48a8-94c1-f192b487e4f6 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:22.102872 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1f39cd-7994-48a8-94c1-f192b487e4f6 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:22.103539 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1f39cd-7994-48a8-94c1-f192b487e4f6 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:22.104679 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-53c49f51-5e44-4d77-aed7-cf3178ba1447 None None] MFA Rules not processed for user `b2c03f0365ee4a26b8c076ae365325b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:38:22.105196 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 983/1960] 10.222.0.15 () {54 vars in 921 bytes} [Mon Oct 31 12:38:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 31 12:38:22.105879 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-53c49f51-5e44-4d77-aed7-cf3178ba1447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:38:22.112499 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5fc9bc-f683-42af-abfa-40c55d8d588f None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:22.112860 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5fc9bc-f683-42af-abfa-40c55d8d588f None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:22.113347 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5fc9bc-f683-42af-abfa-40c55d8d588f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:22.113613 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 978/1961] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:38:22 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:38:22.136406 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-df5fc9bc-f683-42af-abfa-40c55d8d588f None None] MFA Rules not processed for user `9e7777495d7a406c8c50b1ad52e47263`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:38:22.137592 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-df5fc9bc-f683-42af-abfa-40c55d8d588f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:38:22.148346 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 984/1962] 10.222.0.15 () {58 vars in 1018 bytes} [Mon Oct 31 12:38:22 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 31 12:38:22.167831 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c49f51-5e44-4d77-aed7-cf3178ba1447 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:22.179503 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5fc9bc-f683-42af-abfa-40c55d8d588f None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:22.180976 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-df5fc9bc-f683-42af-abfa-40c55d8d588f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:38:22.190312 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c49f51-5e44-4d77-aed7-cf3178ba1447 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:22.197401 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cc9e3f-438e-4a22-8f05-86616d4c918e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e7777495d7a406c8c50b1ad52e47263', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:22.198263 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cc9e3f-438e-4a22-8f05-86616d4c918e service placement] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:22.198538 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cc9e3f-438e-4a22-8f05-86616d4c918e service placement] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:22.198946 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cc9e3f-438e-4a22-8f05-86616d4c918e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:22.201660 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5fc9bc-f683-42af-abfa-40c55d8d588f None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:22.206525 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84fe8c0-ac41-435c-80c2-7c5a46bddb68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:22.207154 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84fe8c0-ac41-435c-80c2-7c5a46bddb68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:22.207470 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84fe8c0-ac41-435c-80c2-7c5a46bddb68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:22.207910 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84fe8c0-ac41-435c-80c2-7c5a46bddb68 admin admin] PATH_INFO: `/v3/users/db2e978bcb824a5e9d593b861c9e3bd7` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:22.209262 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84fe8c0-ac41-435c-80c2-7c5a46bddb68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db2e978bcb824a5e9d593b861c9e3bd7)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:22.211520 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84fe8c0-ac41-435c-80c2-7c5a46bddb68 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:22.214441 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cc9e3f-438e-4a22-8f05-86616d4c918e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:22.216068 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cc9e3f-438e-4a22-8f05-86616d4c918e service placement] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:22.232008 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 979/1963] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:38:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:38:22.270797 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-c84fe8c0-ac41-435c-80c2-7c5a46bddb68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db2e978bcb824a5e9d593b861c9e3bd7'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:22.279236 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-c84fe8c0-ac41-435c-80c2-7c5a46bddb68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db2e978bcb824a5e9d593b861c9e3bd7'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:22.280068 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-c84fe8c0-ac41-435c-80c2-7c5a46bddb68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db2e978bcb824a5e9d593b861c9e3bd7'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:22.280693 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-c84fe8c0-ac41-435c-80c2-7c5a46bddb68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db2e978bcb824a5e9d593b861c9e3bd7'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:22.294191 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-c84fe8c0-ac41-435c-80c2-7c5a46bddb68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db2e978bcb824a5e9d593b861c9e3bd7'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:22.307578 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 985/1964] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:38:22 2022] DELETE /identity/v3/users/db2e978bcb824a5e9d593b861c9e3bd7 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:38:22.454414 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cc9e3f-438e-4a22-8f05-86616d4c918e service placement] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:22.464274 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cc9e3f-438e-4a22-8f05-86616d4c918e service placement] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:22.492148 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84fe8c0-ac41-435c-80c2-7c5a46bddb68 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:22.493646 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-c84fe8c0-ac41-435c-80c2-7c5a46bddb68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:38:22.524772 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091e963e-d53b-4b28-9abb-7ed171e28609 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:22.526139 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091e963e-d53b-4b28-9abb-7ed171e28609 service neutron] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:22.526477 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091e963e-d53b-4b28-9abb-7ed171e28609 service neutron] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:22.526823 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091e963e-d53b-4b28-9abb-7ed171e28609 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:22.528704 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-091e963e-d53b-4b28-9abb-7ed171e28609 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:38:22.539216 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091e963e-d53b-4b28-9abb-7ed171e28609 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:22.540853 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091e963e-d53b-4b28-9abb-7ed171e28609 service neutron] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:22.548978 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 980/1965] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:38:22 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:38:22.567639 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84fe8c0-ac41-435c-80c2-7c5a46bddb68 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:22.572359 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b963a0db-4046-4008-b2ca-33751a68f724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:22.573158 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b963a0db-4046-4008-b2ca-33751a68f724 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:22.573434 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b963a0db-4046-4008-b2ca-33751a68f724 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:22.573702 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b963a0db-4046-4008-b2ca-33751a68f724 admin admin] PATH_INFO: `/v3/projects/7573cb9c503243ab90b1649591d550e6` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:22.574935 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b963a0db-4046-4008-b2ca-33751a68f724 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7573cb9c503243ab90b1649591d550e6)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:22.576875 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b963a0db-4046-4008-b2ca-33751a68f724 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:22.578382 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:38:22.626902 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-b963a0db-4046-4008-b2ca-33751a68f724 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7573cb9c503243ab90b1649591d550e6', 'request_id': 'req-b963a0db-4046-4008-b2ca-33751a68f724'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:22.632935 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-b963a0db-4046-4008-b2ca-33751a68f724 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7573cb9c503243ab90b1649591d550e6', 'request_id': 'req-b963a0db-4046-4008-b2ca-33751a68f724'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:22.639833 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-b963a0db-4046-4008-b2ca-33751a68f724 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7573cb9c503243ab90b1649591d550e6', 'request_id': 'req-b963a0db-4046-4008-b2ca-33751a68f724'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:22.652212 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-b963a0db-4046-4008-b2ca-33751a68f724 admin admin] The token cache is being invalidate because project 7573cb9c503243ab90b1649591d550e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:38:22.652587 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-b963a0db-4046-4008-b2ca-33751a68f724 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:22.653304 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-b963a0db-4046-4008-b2ca-33751a68f724 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:22.654863 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 986/1966] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:38:22 2022] DELETE /identity/v3/projects/7573cb9c503243ab90b1649591d550e6 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:38:22.659747 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afe6c1a-eebf-4a24-89d2-f032fe668976 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:22.660048 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afe6c1a-eebf-4a24-89d2-f032fe668976 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:22.660325 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afe6c1a-eebf-4a24-89d2-f032fe668976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:22.680987 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-1afe6c1a-eebf-4a24-89d2-f032fe668976 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:38:22.683298 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-1afe6c1a-eebf-4a24-89d2-f032fe668976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:38:22.744306 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 981/1967] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:38:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 31 12:38:22.752085 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af75027f-3983-4234-94fa-60c6fffc500d None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:22.752479 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af75027f-3983-4234-94fa-60c6fffc500d None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:22.752816 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af75027f-3983-4234-94fa-60c6fffc500d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:22.772851 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-af75027f-3983-4234-94fa-60c6fffc500d None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:38:22.774065 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-af75027f-3983-4234-94fa-60c6fffc500d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:38:22.783269 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 987/1968] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:38:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:38:22.787716 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afe6c1a-eebf-4a24-89d2-f032fe668976 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:22.797851 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afe6c1a-eebf-4a24-89d2-f032fe668976 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:22.802655 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d046590b-3876-4a0b-8cf5-07f6e6da7b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:22.803330 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d046590b-3876-4a0b-8cf5-07f6e6da7b2b admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:22.803608 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d046590b-3876-4a0b-8cf5-07f6e6da7b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:22.803876 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d046590b-3876-4a0b-8cf5-07f6e6da7b2b admin admin] PATH_INFO: `/v3/domains` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:22.804544 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d046590b-3876-4a0b-8cf5-07f6e6da7b2b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:38:22.804841 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d046590b-3876-4a0b-8cf5-07f6e6da7b2b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:22.806946 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d046590b-3876-4a0b-8cf5-07f6e6da7b2b admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:22.818789 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 982/1969] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Oct 31 12:38:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:38:22.823892 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af75027f-3983-4234-94fa-60c6fffc500d None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:22.833573 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af75027f-3983-4234-94fa-60c6fffc500d None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:22.838968 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d82e26-0bb0-4727-9c0d-aacfe54f248f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:22.838968 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d82e26-0bb0-4727-9c0d-aacfe54f248f admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:22.839370 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d82e26-0bb0-4727-9c0d-aacfe54f248f admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:22.839530 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d82e26-0bb0-4727-9c0d-aacfe54f248f admin admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:22.840148 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d82e26-0bb0-4727-9c0d-aacfe54f248f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:22.843132 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d82e26-0bb0-4727-9c0d-aacfe54f248f admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:22.877805 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 988/1970] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:38:22 2022] POST /identity/v3/projects => generated 403 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:38:22.881731 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d046590b-3876-4a0b-8cf5-07f6e6da7b2b admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:22.881882 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d82e26-0bb0-4727-9c0d-aacfe54f248f admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:22.883685 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-d046590b-3876-4a0b-8cf5-07f6e6da7b2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:38:22.961862 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d82e26-0bb0-4727-9c0d-aacfe54f248f admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:22.965758 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d046590b-3876-4a0b-8cf5-07f6e6da7b2b admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:22.966731 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235fc962-43ea-4be9-a687-4f08bd5246d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:22.967387 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235fc962-43ea-4be9-a687-4f08bd5246d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:22.967674 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235fc962-43ea-4be9-a687-4f08bd5246d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:22.967929 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235fc962-43ea-4be9-a687-4f08bd5246d1 admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:22.968551 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235fc962-43ea-4be9-a687-4f08bd5246d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:22.970434 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235fc962-43ea-4be9-a687-4f08bd5246d1 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:22.974514 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:38:22.991763 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 989/1971] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:38:22 2022] POST /identity/v3/users => generated 302 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:38:22.996413 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235fc962-43ea-4be9-a687-4f08bd5246d1 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:23.006669 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235fc962-43ea-4be9-a687-4f08bd5246d1 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:23.011942 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eaad55-9dcc-467b-8a71-8ef9c78f6521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:23.012543 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eaad55-9dcc-467b-8a71-8ef9c78f6521 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:23.012945 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eaad55-9dcc-467b-8a71-8ef9c78f6521 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:23.013225 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eaad55-9dcc-467b-8a71-8ef9c78f6521 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:23.013860 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eaad55-9dcc-467b-8a71-8ef9c78f6521 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:23.015486 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eaad55-9dcc-467b-8a71-8ef9c78f6521 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:23.028703 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 990/1972] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:38:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 31 12:38:23.029444 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a496ab-0717-4914-9b7e-5208a68555c6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e7777495d7a406c8c50b1ad52e47263', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:23.029798 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eaad55-9dcc-467b-8a71-8ef9c78f6521 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:23.030068 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a496ab-0717-4914-9b7e-5208a68555c6 service placement] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:23.030349 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a496ab-0717-4914-9b7e-5208a68555c6 service placement] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:23.030623 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a496ab-0717-4914-9b7e-5208a68555c6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:23.032553 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-a1a496ab-0717-4914-9b7e-5208a68555c6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:38:23.039972 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eaad55-9dcc-467b-8a71-8ef9c78f6521 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:23.045309 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1cbd2d-10c7-46f2-a127-3f9bd95d2367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:23.045834 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a496ab-0717-4914-9b7e-5208a68555c6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:23.045950 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1cbd2d-10c7-46f2-a127-3f9bd95d2367 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:23.046174 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1cbd2d-10c7-46f2-a127-3f9bd95d2367 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:23.046476 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1cbd2d-10c7-46f2-a127-3f9bd95d2367 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:23.047107 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1cbd2d-10c7-46f2-a127-3f9bd95d2367 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:23.047443 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a496ab-0717-4914-9b7e-5208a68555c6 service placement] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:23.048703 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1cbd2d-10c7-46f2-a127-3f9bd95d2367 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:23.057616 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 983/1973] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:38:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:38:23.059237 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 991/1974] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:38:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:38:23.063435 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a496ab-0717-4914-9b7e-5208a68555c6 service placement] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:23.074320 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a496ab-0717-4914-9b7e-5208a68555c6 service placement] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:23.078897 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2001d125-0396-4dda-9dab-8fa8b1b735f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:23.079563 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2001d125-0396-4dda-9dab-8fa8b1b735f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:23.079842 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2001d125-0396-4dda-9dab-8fa8b1b735f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:23.080114 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2001d125-0396-4dda-9dab-8fa8b1b735f1 admin admin] PATH_INFO: `/v3/projects/4005cd2986394a5980c4d90f1b82c59e/users/3aa2de0852bc40c88083a5a2ed52d838/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:23.094985 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2001d125-0396-4dda-9dab-8fa8b1b735f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4005cd2986394a5980c4d90f1b82c59e, user_id=3aa2de0852bc40c88083a5a2ed52d838, role_id=ca1a5695f8bd499aa0b8cedaeecd2cc6)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:23.097661 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2001d125-0396-4dda-9dab-8fa8b1b735f1 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:23.111083 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 984/1975] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:38:23 2022] PUT /identity/v3/projects/4005cd2986394a5980c4d90f1b82c59e/users/3aa2de0852bc40c88083a5a2ed52d838/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:38:23.116214 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ec3e29-27cf-42b9-80c2-5a647ba183df None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:23.116667 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ec3e29-27cf-42b9-80c2-5a647ba183df None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:23.116761 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ec3e29-27cf-42b9-80c2-5a647ba183df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:23.137169 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-c6ec3e29-27cf-42b9-80c2-5a647ba183df None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:38:23.138375 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-c6ec3e29-27cf-42b9-80c2-5a647ba183df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:38:23.164281 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:38:23.203597 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 992/1976] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:38:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:38:23.213562 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2001d125-0396-4dda-9dab-8fa8b1b735f1 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:23.215539 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-2001d125-0396-4dda-9dab-8fa8b1b735f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:38:23.282364 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2001d125-0396-4dda-9dab-8fa8b1b735f1 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:23.286176 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5e0aea-38d3-427a-8cb8-f275d1245d90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:23.286795 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5e0aea-38d3-427a-8cb8-f275d1245d90 service neutron] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:23.287074 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5e0aea-38d3-427a-8cb8-f275d1245d90 service neutron] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:23.287376 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5e0aea-38d3-427a-8cb8-f275d1245d90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:23.297059 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5e0aea-38d3-427a-8cb8-f275d1245d90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:23.298500 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5e0aea-38d3-427a-8cb8-f275d1245d90 service neutron] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:23.308890 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 985/1977] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:38:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:38:23.935313 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b4f960-9a4a-41c5-97bd-b594a6e12eae None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:23.935601 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b4f960-9a4a-41c5-97bd-b594a6e12eae None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:23.935875 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b4f960-9a4a-41c5-97bd-b594a6e12eae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:23.967555 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-05b4f960-9a4a-41c5-97bd-b594a6e12eae None None] MFA Rules not processed for user `3aa2de0852bc40c88083a5a2ed52d838`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:38:23.969697 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-05b4f960-9a4a-41c5-97bd-b594a6e12eae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:38:24.054557 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 993/1978] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:38:23 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:38:24.056004 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5e0aea-38d3-427a-8cb8-f275d1245d90 service neutron] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:24.066321 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5e0aea-38d3-427a-8cb8-f275d1245d90 service neutron] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:24.070972 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be487322-7f4f-4f53-b930-3e8b025d0f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:24.071696 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be487322-7f4f-4f53-b930-3e8b025d0f4f admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:24.072077 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be487322-7f4f-4f53-b930-3e8b025d0f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:24.072415 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be487322-7f4f-4f53-b930-3e8b025d0f4f admin admin] PATH_INFO: `/v3/projects` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:24.073153 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be487322-7f4f-4f53-b930-3e8b025d0f4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:24.075148 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be487322-7f4f-4f53-b930-3e8b025d0f4f admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:24.103083 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 986/1979] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:38:24 2022] POST /identity/v3/projects => generated 403 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:38:24.107653 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b4f960-9a4a-41c5-97bd-b594a6e12eae None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:24.118177 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b4f960-9a4a-41c5-97bd-b594a6e12eae None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:24.177144 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32711f6-18e1-4d61-abde-8c3bb54e601b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:24.177757 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32711f6-18e1-4d61-abde-8c3bb54e601b admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:24.178040 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32711f6-18e1-4d61-abde-8c3bb54e601b admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:24.178311 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32711f6-18e1-4d61-abde-8c3bb54e601b admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:24.178939 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32711f6-18e1-4d61-abde-8c3bb54e601b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:24.180879 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32711f6-18e1-4d61-abde-8c3bb54e601b admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:24.203830 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 994/1980] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:38:24 2022] POST /identity/v3/users => generated 308 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 31 12:38:24.207951 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be487322-7f4f-4f53-b930-3e8b025d0f4f admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:24.218903 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be487322-7f4f-4f53-b930-3e8b025d0f4f admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:24.224774 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f78d96-c051-4529-ba50-8cea18660ce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:24.225347 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f78d96-c051-4529-ba50-8cea18660ce0 admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:24.225626 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f78d96-c051-4529-ba50-8cea18660ce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:24.225901 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f78d96-c051-4529-ba50-8cea18660ce0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:24.226557 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f78d96-c051-4529-ba50-8cea18660ce0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:24.228379 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f78d96-c051-4529-ba50-8cea18660ce0 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:24.240641 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 987/1981] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:38:24 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:38:24.245695 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32711f6-18e1-4d61-abde-8c3bb54e601b admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:24.256293 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32711f6-18e1-4d61-abde-8c3bb54e601b admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:24.261531 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7f512d-58f2-4bb3-a71b-eedff0dba9be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:24.262500 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7f512d-58f2-4bb3-a71b-eedff0dba9be admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:24.262953 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7f512d-58f2-4bb3-a71b-eedff0dba9be admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:24.263419 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7f512d-58f2-4bb3-a71b-eedff0dba9be admin admin] PATH_INFO: `/v3/projects/c8a3ee1772f544c99bc2a8678151dbfa/users/cff57f111f6a4253ba3262ab6787a9a9/roles/176c07b4fe3b4537bdfeb4be21b379c5` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:24.280105 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7f512d-58f2-4bb3-a71b-eedff0dba9be admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8a3ee1772f544c99bc2a8678151dbfa, user_id=cff57f111f6a4253ba3262ab6787a9a9, role_id=176c07b4fe3b4537bdfeb4be21b379c5)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:24.282709 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7f512d-58f2-4bb3-a71b-eedff0dba9be admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:24.296529 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 995/1982] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:38:24 2022] PUT /identity/v3/projects/c8a3ee1772f544c99bc2a8678151dbfa/users/cff57f111f6a4253ba3262ab6787a9a9/roles/176c07b4fe3b4537bdfeb4be21b379c5 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:38:24.300893 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f78d96-c051-4529-ba50-8cea18660ce0 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:24.312979 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f78d96-c051-4529-ba50-8cea18660ce0 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:24.365192 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:38:24.381953 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720ee069-3e9d-4f7a-990f-4b1d2987b9f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:24.382573 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720ee069-3e9d-4f7a-990f-4b1d2987b9f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:24.382852 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720ee069-3e9d-4f7a-990f-4b1d2987b9f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:24.383217 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720ee069-3e9d-4f7a-990f-4b1d2987b9f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:24.383990 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720ee069-3e9d-4f7a-990f-4b1d2987b9f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:24.386277 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720ee069-3e9d-4f7a-990f-4b1d2987b9f2 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:24.399607 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 988/1983] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:38:24 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 31 12:38:24.404071 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7f512d-58f2-4bb3-a71b-eedff0dba9be admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:24.414385 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7f512d-58f2-4bb3-a71b-eedff0dba9be admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:24.419649 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282ba1c2-5c76-430e-ac85-9cd1a10790bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:24.420326 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282ba1c2-5c76-430e-ac85-9cd1a10790bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:24.420605 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282ba1c2-5c76-430e-ac85-9cd1a10790bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:24.420879 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282ba1c2-5c76-430e-ac85-9cd1a10790bc admin admin] PATH_INFO: `/v3/projects/c8a3ee1772f544c99bc2a8678151dbfa/users/cff57f111f6a4253ba3262ab6787a9a9/roles/176c07b4fe3b4537bdfeb4be21b379c5` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:24.424946 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282ba1c2-5c76-430e-ac85-9cd1a10790bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8a3ee1772f544c99bc2a8678151dbfa, user_id=cff57f111f6a4253ba3262ab6787a9a9, role_id=176c07b4fe3b4537bdfeb4be21b379c5)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:24.429105 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282ba1c2-5c76-430e-ac85-9cd1a10790bc admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:24.440404 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 996/1984] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:38:24 2022] PUT /identity/v3/projects/c8a3ee1772f544c99bc2a8678151dbfa/users/cff57f111f6a4253ba3262ab6787a9a9/roles/176c07b4fe3b4537bdfeb4be21b379c5 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:38:25.304521 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720ee069-3e9d-4f7a-990f-4b1d2987b9f2 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:25.344810 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f5a41d-64aa-4a5e-aa4b-c565051fc1b7 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:25.345346 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f5a41d-64aa-4a5e-aa4b-c565051fc1b7 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:25.345346 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f5a41d-64aa-4a5e-aa4b-c565051fc1b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:25.372243 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:38:25.382529 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-58f5a41d-64aa-4a5e-aa4b-c565051fc1b7 None None] MFA Rules not processed for user `cff57f111f6a4253ba3262ab6787a9a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:38:25.385422 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-58f5a41d-64aa-4a5e-aa4b-c565051fc1b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:38:25.400087 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720ee069-3e9d-4f7a-990f-4b1d2987b9f2 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:25.463051 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd24bf5-dd80-49c3-8a47-d88366a11957 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:25.463691 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd24bf5-dd80-49c3-8a47-d88366a11957 service neutron] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:25.464135 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd24bf5-dd80-49c3-8a47-d88366a11957 service neutron] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:25.464562 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd24bf5-dd80-49c3-8a47-d88366a11957 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:25.466645 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-9cd24bf5-dd80-49c3-8a47-d88366a11957 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:38:25.476640 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd24bf5-dd80-49c3-8a47-d88366a11957 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:25.477707 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 997/1985] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:38:25 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 31 12:38:25.483867 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd24bf5-dd80-49c3-8a47-d88366a11957 service neutron] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:25.493103 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 989/1986] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:38:25 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 31 12:38:26.638512 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f5a41d-64aa-4a5e-aa4b-c565051fc1b7 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:26.640135 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-58f5a41d-64aa-4a5e-aa4b-c565051fc1b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:38:26.741608 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f5a41d-64aa-4a5e-aa4b-c565051fc1b7 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:26.749091 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e763a6a7-7c98-4259-a000-624a219185c0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99889b1dab0b471d9c77887cffefc652', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:26.749821 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e763a6a7-7c98-4259-a000-624a219185c0 service nova] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:26.750171 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e763a6a7-7c98-4259-a000-624a219185c0 service nova] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:26.750518 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e763a6a7-7c98-4259-a000-624a219185c0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:26.752368 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-e763a6a7-7c98-4259-a000-624a219185c0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:38:26.765299 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e763a6a7-7c98-4259-a000-624a219185c0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:26.767120 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e763a6a7-7c98-4259-a000-624a219185c0 service nova] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:26.779864 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 998/1987] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:38:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:38:27.000438 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd24bf5-dd80-49c3-8a47-d88366a11957 service neutron] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:27.069727 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd24bf5-dd80-49c3-8a47-d88366a11957 service neutron] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:27.074502 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202690d0-4a52-4ce2-9f8a-ca8a1827440a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:27.075239 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202690d0-4a52-4ce2-9f8a-ca8a1827440a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:27.075584 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202690d0-4a52-4ce2-9f8a-ca8a1827440a admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:27.075919 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202690d0-4a52-4ce2-9f8a-ca8a1827440a admin admin] PATH_INFO: `/v3/users/3aa2de0852bc40c88083a5a2ed52d838` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:27.077114 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202690d0-4a52-4ce2-9f8a-ca8a1827440a admin admin] RBAC: Authorizing `identity:delete_user(user_id=3aa2de0852bc40c88083a5a2ed52d838)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:27.079077 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202690d0-4a52-4ce2-9f8a-ca8a1827440a admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:27.128508 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:38:27.130448 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-202690d0-4a52-4ce2-9f8a-ca8a1827440a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3aa2de0852bc40c88083a5a2ed52d838'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:27.138927 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-202690d0-4a52-4ce2-9f8a-ca8a1827440a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3aa2de0852bc40c88083a5a2ed52d838'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:27.139589 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-202690d0-4a52-4ce2-9f8a-ca8a1827440a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3aa2de0852bc40c88083a5a2ed52d838'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:27.152825 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-202690d0-4a52-4ce2-9f8a-ca8a1827440a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3aa2de0852bc40c88083a5a2ed52d838'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:27.159812 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-202690d0-4a52-4ce2-9f8a-ca8a1827440a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3aa2de0852bc40c88083a5a2ed52d838'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:27.166376 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 990/1988] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:38:26 2022] DELETE /identity/v3/users/3aa2de0852bc40c88083a5a2ed52d838 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:38:27.336971 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e763a6a7-7c98-4259-a000-624a219185c0 service nova] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:27.349343 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e763a6a7-7c98-4259-a000-624a219185c0 service nova] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:27.412593 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c046c27-7d72-48ee-82eb-a362cb0fcbca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:27.413231 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c046c27-7d72-48ee-82eb-a362cb0fcbca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:27.413513 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c046c27-7d72-48ee-82eb-a362cb0fcbca admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:27.413783 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c046c27-7d72-48ee-82eb-a362cb0fcbca admin admin] PATH_INFO: `/v3/projects/4005cd2986394a5980c4d90f1b82c59e` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:27.414921 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c046c27-7d72-48ee-82eb-a362cb0fcbca admin admin] RBAC: Authorizing `identity:delete_project(project_id=4005cd2986394a5980c4d90f1b82c59e)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:27.417292 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c046c27-7d72-48ee-82eb-a362cb0fcbca admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:27.441035 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:38:27.465418 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-4c046c27-7d72-48ee-82eb-a362cb0fcbca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4005cd2986394a5980c4d90f1b82c59e', 'request_id': 'req-4c046c27-7d72-48ee-82eb-a362cb0fcbca'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:27.471576 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-4c046c27-7d72-48ee-82eb-a362cb0fcbca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4005cd2986394a5980c4d90f1b82c59e', 'request_id': 'req-4c046c27-7d72-48ee-82eb-a362cb0fcbca'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:27.479461 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-4c046c27-7d72-48ee-82eb-a362cb0fcbca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4005cd2986394a5980c4d90f1b82c59e', 'request_id': 'req-4c046c27-7d72-48ee-82eb-a362cb0fcbca'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:27.489683 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-4c046c27-7d72-48ee-82eb-a362cb0fcbca admin admin] The token cache is being invalidate because project 4005cd2986394a5980c4d90f1b82c59e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:38:27.489994 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-4c046c27-7d72-48ee-82eb-a362cb0fcbca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:27.490620 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-4c046c27-7d72-48ee-82eb-a362cb0fcbca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:27.492154 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 999/1989] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:38:27 2022] DELETE /identity/v3/projects/4005cd2986394a5980c4d90f1b82c59e => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:38:27.496780 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202690d0-4a52-4ce2-9f8a-ca8a1827440a admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:27.498142 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-202690d0-4a52-4ce2-9f8a-ca8a1827440a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:38:27.510968 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202690d0-4a52-4ce2-9f8a-ca8a1827440a admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:27.567135 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f184778-65b1-4bdb-988a-75b1a51193a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:27.567796 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f184778-65b1-4bdb-988a-75b1a51193a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:27.568074 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f184778-65b1-4bdb-988a-75b1a51193a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:27.568345 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f184778-65b1-4bdb-988a-75b1a51193a9 admin admin] PATH_INFO: `/v3/users/cff57f111f6a4253ba3262ab6787a9a9` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:27.569457 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f184778-65b1-4bdb-988a-75b1a51193a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cff57f111f6a4253ba3262ab6787a9a9)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:27.571389 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f184778-65b1-4bdb-988a-75b1a51193a9 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:27.625953 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-8f184778-65b1-4bdb-988a-75b1a51193a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cff57f111f6a4253ba3262ab6787a9a9'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:27.633668 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-8f184778-65b1-4bdb-988a-75b1a51193a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cff57f111f6a4253ba3262ab6787a9a9'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:27.634364 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-8f184778-65b1-4bdb-988a-75b1a51193a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cff57f111f6a4253ba3262ab6787a9a9'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:27.647072 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-8f184778-65b1-4bdb-988a-75b1a51193a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cff57f111f6a4253ba3262ab6787a9a9'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:27.654394 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-8f184778-65b1-4bdb-988a-75b1a51193a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cff57f111f6a4253ba3262ab6787a9a9'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:27.660485 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 991/1990] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:38:27 2022] DELETE /identity/v3/users/cff57f111f6a4253ba3262ab6787a9a9 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:38:27.831703 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c046c27-7d72-48ee-82eb-a362cb0fcbca admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:27.833282 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-4c046c27-7d72-48ee-82eb-a362cb0fcbca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:38:27.845759 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c046c27-7d72-48ee-82eb-a362cb0fcbca admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:27.902196 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b00b9b5-0734-4e14-9966-029f7acb98d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:27.902728 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b00b9b5-0734-4e14-9966-029f7acb98d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:27.902999 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b00b9b5-0734-4e14-9966-029f7acb98d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:27.903302 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b00b9b5-0734-4e14-9966-029f7acb98d9 admin admin] PATH_INFO: `/v3/projects/c8a3ee1772f544c99bc2a8678151dbfa` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:27.904418 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b00b9b5-0734-4e14-9966-029f7acb98d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8a3ee1772f544c99bc2a8678151dbfa)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:27.906291 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b00b9b5-0734-4e14-9966-029f7acb98d9 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:27.952008 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-0b00b9b5-0734-4e14-9966-029f7acb98d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8a3ee1772f544c99bc2a8678151dbfa', 'request_id': 'req-0b00b9b5-0734-4e14-9966-029f7acb98d9'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:27.957979 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-0b00b9b5-0734-4e14-9966-029f7acb98d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8a3ee1772f544c99bc2a8678151dbfa', 'request_id': 'req-0b00b9b5-0734-4e14-9966-029f7acb98d9'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:27.965412 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-0b00b9b5-0734-4e14-9966-029f7acb98d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8a3ee1772f544c99bc2a8678151dbfa', 'request_id': 'req-0b00b9b5-0734-4e14-9966-029f7acb98d9'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:27.975832 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-0b00b9b5-0734-4e14-9966-029f7acb98d9 admin admin] The token cache is being invalidate because project c8a3ee1772f544c99bc2a8678151dbfa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:38:27.976122 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-0b00b9b5-0734-4e14-9966-029f7acb98d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:27.976736 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-0b00b9b5-0734-4e14-9966-029f7acb98d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:27.978403 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1000/1991] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:38:27 2022] DELETE /identity/v3/projects/c8a3ee1772f544c99bc2a8678151dbfa => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:38:27.983232 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66d89a9-a65d-44ec-b83d-829d8dae21f3 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:27.983519 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66d89a9-a65d-44ec-b83d-829d8dae21f3 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:27.983795 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66d89a9-a65d-44ec-b83d-829d8dae21f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:28.003737 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-d66d89a9-a65d-44ec-b83d-829d8dae21f3 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:38:28.004936 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-d66d89a9-a65d-44ec-b83d-829d8dae21f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:38:28.046202 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:38:28.070174 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 992/1992] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:38:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:38:28.078463 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a84f8a-b669-42cb-8389-1034026b533d None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:28.078753 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a84f8a-b669-42cb-8389-1034026b533d None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:28.079027 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a84f8a-b669-42cb-8389-1034026b533d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:28.098809 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-d7a84f8a-b669-42cb-8389-1034026b533d None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:38:28.100041 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-d7a84f8a-b669-42cb-8389-1034026b533d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:38:28.112739 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1001/1993] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:38:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 31 12:38:28.117241 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66d89a9-a65d-44ec-b83d-829d8dae21f3 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:28.128533 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66d89a9-a65d-44ec-b83d-829d8dae21f3 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:28.133229 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c94552e-dc3d-4df9-a20c-8eb89e7859d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:28.133876 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c94552e-dc3d-4df9-a20c-8eb89e7859d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:28.134303 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c94552e-dc3d-4df9-a20c-8eb89e7859d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:28.134657 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c94552e-dc3d-4df9-a20c-8eb89e7859d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:28.135271 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c94552e-dc3d-4df9-a20c-8eb89e7859d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:38:28.135583 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c94552e-dc3d-4df9-a20c-8eb89e7859d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:28.137208 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c94552e-dc3d-4df9-a20c-8eb89e7859d0 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:28.152506 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 993/1994] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Oct 31 12:38:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 31 12:38:28.157283 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a84f8a-b669-42cb-8389-1034026b533d None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:28.168078 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a84f8a-b669-42cb-8389-1034026b533d None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:28.173827 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5d212a-3608-41b1-ab93-a9267abc041f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:28.174466 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5d212a-3608-41b1-ab93-a9267abc041f admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:28.174730 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5d212a-3608-41b1-ab93-a9267abc041f admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:28.175032 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5d212a-3608-41b1-ab93-a9267abc041f admin admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:28.175743 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5d212a-3608-41b1-ab93-a9267abc041f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:28.177974 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5d212a-3608-41b1-ab93-a9267abc041f admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:28.211238 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1002/1995] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:38:28 2022] POST /identity/v3/projects => generated 381 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:38:28.214300 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c94552e-dc3d-4df9-a20c-8eb89e7859d0 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:28.225175 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c94552e-dc3d-4df9-a20c-8eb89e7859d0 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:28.293800 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ddfbec-53b3-41b3-93ff-2e288d67143c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:28.294512 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ddfbec-53b3-41b3-93ff-2e288d67143c admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:28.294861 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ddfbec-53b3-41b3-93ff-2e288d67143c admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:28.295226 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ddfbec-53b3-41b3-93ff-2e288d67143c admin admin] PATH_INFO: `/v3/users` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:28.296130 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ddfbec-53b3-41b3-93ff-2e288d67143c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:28.297908 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ddfbec-53b3-41b3-93ff-2e288d67143c admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:28.323574 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 994/1996] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:38:28 2022] POST /identity/v3/users => generated 291 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:38:28.327962 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5d212a-3608-41b1-ab93-a9267abc041f admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:28.348545 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5d212a-3608-41b1-ab93-a9267abc041f admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:28.353815 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a3a81a-234c-48e3-a77a-99bd33da9761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:28.354520 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a3a81a-234c-48e3-a77a-99bd33da9761 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:28.354801 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a3a81a-234c-48e3-a77a-99bd33da9761 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:28.355074 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a3a81a-234c-48e3-a77a-99bd33da9761 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:28.355751 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a3a81a-234c-48e3-a77a-99bd33da9761 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:28.357437 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a3a81a-234c-48e3-a77a-99bd33da9761 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:28.367920 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1003/1997] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:38:28 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:38:28.373006 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ddfbec-53b3-41b3-93ff-2e288d67143c admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:28.383181 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ddfbec-53b3-41b3-93ff-2e288d67143c admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:28.387978 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f6cf48-4e30-45cc-b5ca-f2e1c8b98984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:28.388580 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f6cf48-4e30-45cc-b5ca-f2e1c8b98984 admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:28.388860 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f6cf48-4e30-45cc-b5ca-f2e1c8b98984 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:28.389135 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f6cf48-4e30-45cc-b5ca-f2e1c8b98984 admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:28.389776 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f6cf48-4e30-45cc-b5ca-f2e1c8b98984 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:28.391395 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f6cf48-4e30-45cc-b5ca-f2e1c8b98984 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:28.403515 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 995/1998] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:38:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:38:28.407529 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a3a81a-234c-48e3-a77a-99bd33da9761 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:28.413033 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:38:28.421382 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a3a81a-234c-48e3-a77a-99bd33da9761 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:28.426072 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8c9ce1-ba46-49f9-9dbc-95f729bf59a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:28.426762 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8c9ce1-ba46-49f9-9dbc-95f729bf59a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:28.427467 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8c9ce1-ba46-49f9-9dbc-95f729bf59a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:28.428127 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8c9ce1-ba46-49f9-9dbc-95f729bf59a8 admin admin] PATH_INFO: `/v3/projects/30e029902ef7448883710930b23ef048/users/db8f776debf54b5ca62fafe2c8277c29/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:28.451848 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8c9ce1-ba46-49f9-9dbc-95f729bf59a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30e029902ef7448883710930b23ef048, user_id=db8f776debf54b5ca62fafe2c8277c29, role_id=ca1a5695f8bd499aa0b8cedaeecd2cc6)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:28.454602 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8c9ce1-ba46-49f9-9dbc-95f729bf59a8 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:28.470089 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1004/1999] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:38:28 2022] PUT /identity/v3/projects/30e029902ef7448883710930b23ef048/users/db8f776debf54b5ca62fafe2c8277c29/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:38:28.481657 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f6590c-87cd-4435-af9c-1fe30450fea7 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:28.481946 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f6590c-87cd-4435-af9c-1fe30450fea7 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:28.482220 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f6590c-87cd-4435-af9c-1fe30450fea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:28.514027 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-59f6590c-87cd-4435-af9c-1fe30450fea7 None None] MFA Rules not processed for user `db8f776debf54b5ca62fafe2c8277c29`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:38:28.515282 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-59f6590c-87cd-4435-af9c-1fe30450fea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:38:28.586728 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 996/2000] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:38:28 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:38:28.591724 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8c9ce1-ba46-49f9-9dbc-95f729bf59a8 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:28.656093 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8c9ce1-ba46-49f9-9dbc-95f729bf59a8 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:28.660825 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120f96f9-f9e9-4364-9696-9c41e4f56c1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:28.661440 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120f96f9-f9e9-4364-9696-9c41e4f56c1a admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:28.661723 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120f96f9-f9e9-4364-9696-9c41e4f56c1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:28.661993 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120f96f9-f9e9-4364-9696-9c41e4f56c1a admin admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:28.662627 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120f96f9-f9e9-4364-9696-9c41e4f56c1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:28.664579 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120f96f9-f9e9-4364-9696-9c41e4f56c1a admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:28.692663 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1005/2001] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:38:28 2022] POST /identity/v3/projects => generated 381 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 31 12:38:28.697344 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f6590c-87cd-4435-af9c-1fe30450fea7 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:28.709211 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f6590c-87cd-4435-af9c-1fe30450fea7 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:28.743216 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:38:28.772024 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c38440-c0a7-49a7-9960-737cbb608f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:28.772639 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c38440-c0a7-49a7-9960-737cbb608f89 admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:28.772920 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c38440-c0a7-49a7-9960-737cbb608f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:28.773187 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c38440-c0a7-49a7-9960-737cbb608f89 admin admin] PATH_INFO: `/v3/users` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:28.773810 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c38440-c0a7-49a7-9960-737cbb608f89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:28.776742 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c38440-c0a7-49a7-9960-737cbb608f89 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:28.802526 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 997/2002] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:38:28 2022] POST /identity/v3/users => generated 297 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:38:28.803856 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120f96f9-f9e9-4364-9696-9c41e4f56c1a admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:28.814677 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120f96f9-f9e9-4364-9696-9c41e4f56c1a admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:28.819801 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b872c6a6-d71b-47d7-8d68-63e9dac9f0db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:28.820334 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b872c6a6-d71b-47d7-8d68-63e9dac9f0db admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:28.820614 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b872c6a6-d71b-47d7-8d68-63e9dac9f0db admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:28.820880 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b872c6a6-d71b-47d7-8d68-63e9dac9f0db admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:28.821537 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b872c6a6-d71b-47d7-8d68-63e9dac9f0db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:28.823990 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b872c6a6-d71b-47d7-8d68-63e9dac9f0db admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:28.834797 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1006/2003] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:38:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:38:28.839812 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c38440-c0a7-49a7-9960-737cbb608f89 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:28.849736 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c38440-c0a7-49a7-9960-737cbb608f89 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:28.855283 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef92191-8b9d-4746-8484-7fccd3e93c87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:28.856042 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef92191-8b9d-4746-8484-7fccd3e93c87 admin admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:28.856317 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef92191-8b9d-4746-8484-7fccd3e93c87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:28.856585 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef92191-8b9d-4746-8484-7fccd3e93c87 admin admin] PATH_INFO: `/v3/projects/f0ed898e88cd410082c5c4fccb997b91/users/d11788b1b4424adc94264dc56e9c1b96/roles/176c07b4fe3b4537bdfeb4be21b379c5` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:28.869979 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b872c6a6-d71b-47d7-8d68-63e9dac9f0db admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:28.871757 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-b872c6a6-d71b-47d7-8d68-63e9dac9f0db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:38:28.885987 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef92191-8b9d-4746-8484-7fccd3e93c87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0ed898e88cd410082c5c4fccb997b91, user_id=d11788b1b4424adc94264dc56e9c1b96, role_id=176c07b4fe3b4537bdfeb4be21b379c5)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:28.888706 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef92191-8b9d-4746-8484-7fccd3e93c87 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:28.902743 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 998/2004] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:38:28 2022] PUT /identity/v3/projects/f0ed898e88cd410082c5c4fccb997b91/users/d11788b1b4424adc94264dc56e9c1b96/roles/176c07b4fe3b4537bdfeb4be21b379c5 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:38:28.906944 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef92191-8b9d-4746-8484-7fccd3e93c87 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:28.951844 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b872c6a6-d71b-47d7-8d68-63e9dac9f0db admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:28.972186 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef92191-8b9d-4746-8484-7fccd3e93c87 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:28.976857 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5bcb01-0229-4b25-9cfd-1288801d63a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:28.977467 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5bcb01-0229-4b25-9cfd-1288801d63a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:28.978292 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5bcb01-0229-4b25-9cfd-1288801d63a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:28.979099 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5bcb01-0229-4b25-9cfd-1288801d63a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:28.979781 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5bcb01-0229-4b25-9cfd-1288801d63a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:28.981371 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5bcb01-0229-4b25-9cfd-1288801d63a2 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:28.991366 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 999/2005] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:38:28 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 31 12:38:28.995913 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5bcb01-0229-4b25-9cfd-1288801d63a2 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:28.998555 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:38:29.006121 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5bcb01-0229-4b25-9cfd-1288801d63a2 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:29.011366 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f385e950-209a-4d32-8d1f-5edc163e1acd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:29.012003 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f385e950-209a-4d32-8d1f-5edc163e1acd admin admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:29.012277 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f385e950-209a-4d32-8d1f-5edc163e1acd admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:29.012562 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f385e950-209a-4d32-8d1f-5edc163e1acd admin admin] PATH_INFO: `/v3/projects/f0ed898e88cd410082c5c4fccb997b91/users/d11788b1b4424adc94264dc56e9c1b96/roles/176c07b4fe3b4537bdfeb4be21b379c5` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:29.014417 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f385e950-209a-4d32-8d1f-5edc163e1acd admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0ed898e88cd410082c5c4fccb997b91, user_id=d11788b1b4424adc94264dc56e9c1b96, role_id=176c07b4fe3b4537bdfeb4be21b379c5)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:29.017055 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f385e950-209a-4d32-8d1f-5edc163e1acd admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:29.017636 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8303cc-89e9-416d-a1f6-dcaf1985c807 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99889b1dab0b471d9c77887cffefc652', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:29.018248 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8303cc-89e9-416d-a1f6-dcaf1985c807 service nova] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:29.018529 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8303cc-89e9-416d-a1f6-dcaf1985c807 service nova] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:29.018799 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8303cc-89e9-416d-a1f6-dcaf1985c807 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:29.020917 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-2a8303cc-89e9-416d-a1f6-dcaf1985c807 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:38:29.035731 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1000/2006] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:38:28 2022] PUT /identity/v3/projects/f0ed898e88cd410082c5c4fccb997b91/users/d11788b1b4424adc94264dc56e9c1b96/roles/176c07b4fe3b4537bdfeb4be21b379c5 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 31 12:38:29.046554 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0974cfb3-38c8-49af-a558-0b35a11e7f79 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:29.046852 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0974cfb3-38c8-49af-a558-0b35a11e7f79 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:29.047099 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0974cfb3-38c8-49af-a558-0b35a11e7f79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:29.075220 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-0974cfb3-38c8-49af-a558-0b35a11e7f79 None None] MFA Rules not processed for user `d11788b1b4424adc94264dc56e9c1b96`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:38:29.076435 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-0974cfb3-38c8-49af-a558-0b35a11e7f79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:38:29.088898 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8303cc-89e9-416d-a1f6-dcaf1985c807 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:29.090548 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8303cc-89e9-416d-a1f6-dcaf1985c807 service nova] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:29.099814 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1007/2007] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:38:28 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:38:29.147714 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1001/2008] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:38:29 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 31 12:38:29.160096 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8303cc-89e9-416d-a1f6-dcaf1985c807 service nova] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:29.161439 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-2a8303cc-89e9-416d-a1f6-dcaf1985c807 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:38:29.218133 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8303cc-89e9-416d-a1f6-dcaf1985c807 service nova] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:29.268125 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d806551b-2336-449b-8e12-e68d7af7c812 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d6d7ea077d49e8a66c6c6762499702', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:29.268848 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d806551b-2336-449b-8e12-e68d7af7c812 service cinder] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:29.269376 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d806551b-2336-449b-8e12-e68d7af7c812 service cinder] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:29.269642 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d806551b-2336-449b-8e12-e68d7af7c812 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:29.279512 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d806551b-2336-449b-8e12-e68d7af7c812 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:29.280931 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d806551b-2336-449b-8e12-e68d7af7c812 service cinder] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:29.291044 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1008/2009] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:38:29 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 31 12:38:29.610295 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0974cfb3-38c8-49af-a558-0b35a11e7f79 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:29.620152 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0974cfb3-38c8-49af-a558-0b35a11e7f79 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:29.624736 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0555f3-42c9-4312-bf9d-8fb7711ba6d2 tempest-VolumeQuotaClassesTest-1046517315 tempest-VolumeQuotaClassesTest-1046517315-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd11788b1b4424adc94264dc56e9c1b96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0ed898e88cd410082c5c4fccb997b91', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:29.625335 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0555f3-42c9-4312-bf9d-8fb7711ba6d2 tempest-VolumeQuotaClassesTest-1046517315 tempest-VolumeQuotaClassesTest-1046517315-project-admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:29.625620 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0555f3-42c9-4312-bf9d-8fb7711ba6d2 tempest-VolumeQuotaClassesTest-1046517315 tempest-VolumeQuotaClassesTest-1046517315-project-admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:29.625895 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0555f3-42c9-4312-bf9d-8fb7711ba6d2 tempest-VolumeQuotaClassesTest-1046517315 tempest-VolumeQuotaClassesTest-1046517315-project-admin] PATH_INFO: `/v3/domains` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:29.626564 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0555f3-42c9-4312-bf9d-8fb7711ba6d2 tempest-VolumeQuotaClassesTest-1046517315 tempest-VolumeQuotaClassesTest-1046517315-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:38:29.626865 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0555f3-42c9-4312-bf9d-8fb7711ba6d2 tempest-VolumeQuotaClassesTest-1046517315 tempest-VolumeQuotaClassesTest-1046517315-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:29.628496 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0555f3-42c9-4312-bf9d-8fb7711ba6d2 tempest-VolumeQuotaClassesTest-1046517315 tempest-VolumeQuotaClassesTest-1046517315-project-admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:29.636009 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:38:29.645036 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1002/2010] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Oct 31 12:38:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:38:29.646086 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d806551b-2336-449b-8e12-e68d7af7c812 service cinder] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:29.655933 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d806551b-2336-449b-8e12-e68d7af7c812 service cinder] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:29.661008 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1753a6e1-30e0-4f28-a7dc-d28e3b212f2d tempest-VolumeQuotaClassesTest-1046517315 tempest-VolumeQuotaClassesTest-1046517315-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd11788b1b4424adc94264dc56e9c1b96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0ed898e88cd410082c5c4fccb997b91', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:29.661588 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1753a6e1-30e0-4f28-a7dc-d28e3b212f2d tempest-VolumeQuotaClassesTest-1046517315 tempest-VolumeQuotaClassesTest-1046517315-project-admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:29.661897 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1753a6e1-30e0-4f28-a7dc-d28e3b212f2d tempest-VolumeQuotaClassesTest-1046517315 tempest-VolumeQuotaClassesTest-1046517315-project-admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:29.662172 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1753a6e1-30e0-4f28-a7dc-d28e3b212f2d tempest-VolumeQuotaClassesTest-1046517315 tempest-VolumeQuotaClassesTest-1046517315-project-admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:29.662803 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1753a6e1-30e0-4f28-a7dc-d28e3b212f2d tempest-VolumeQuotaClassesTest-1046517315 tempest-VolumeQuotaClassesTest-1046517315-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:29.664729 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1753a6e1-30e0-4f28-a7dc-d28e3b212f2d tempest-VolumeQuotaClassesTest-1046517315 tempest-VolumeQuotaClassesTest-1046517315-project-admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:29.692641 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1009/2011] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:38:29 2022] POST /identity/v3/projects => generated 353 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 31 12:38:29.697384 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0555f3-42c9-4312-bf9d-8fb7711ba6d2 tempest-VolumeQuotaClassesTest-1046517315 tempest-VolumeQuotaClassesTest-1046517315-project-admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:29.707372 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0555f3-42c9-4312-bf9d-8fb7711ba6d2 tempest-VolumeQuotaClassesTest-1046517315 tempest-VolumeQuotaClassesTest-1046517315-project-admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:29.763620 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ace8c8-3351-4ea5-91fe-b1b63186050c tempest-VolumeQuotaClassesTest-1046517315 tempest-VolumeQuotaClassesTest-1046517315-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd11788b1b4424adc94264dc56e9c1b96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0ed898e88cd410082c5c4fccb997b91', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:29.764221 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ace8c8-3351-4ea5-91fe-b1b63186050c tempest-VolumeQuotaClassesTest-1046517315 tempest-VolumeQuotaClassesTest-1046517315-project-admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:29.764502 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ace8c8-3351-4ea5-91fe-b1b63186050c tempest-VolumeQuotaClassesTest-1046517315 tempest-VolumeQuotaClassesTest-1046517315-project-admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:29.764778 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ace8c8-3351-4ea5-91fe-b1b63186050c tempest-VolumeQuotaClassesTest-1046517315 tempest-VolumeQuotaClassesTest-1046517315-project-admin] PATH_INFO: `/v3/domains` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:29.765412 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ace8c8-3351-4ea5-91fe-b1b63186050c tempest-VolumeQuotaClassesTest-1046517315 tempest-VolumeQuotaClassesTest-1046517315-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:38:29.765716 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ace8c8-3351-4ea5-91fe-b1b63186050c tempest-VolumeQuotaClassesTest-1046517315 tempest-VolumeQuotaClassesTest-1046517315-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:29.767315 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ace8c8-3351-4ea5-91fe-b1b63186050c tempest-VolumeQuotaClassesTest-1046517315 tempest-VolumeQuotaClassesTest-1046517315-project-admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:29.779813 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1003/2012] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Oct 31 12:38:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:38:29.815594 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1753a6e1-30e0-4f28-a7dc-d28e3b212f2d tempest-VolumeQuotaClassesTest-1046517315 tempest-VolumeQuotaClassesTest-1046517315-project-admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:29.826198 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1753a6e1-30e0-4f28-a7dc-d28e3b212f2d tempest-VolumeQuotaClassesTest-1046517315 tempest-VolumeQuotaClassesTest-1046517315-project-admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:29.830704 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b842ec6-0052-4697-885a-5f4ceca1e84a tempest-VolumeQuotaClassesTest-1046517315 tempest-VolumeQuotaClassesTest-1046517315-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd11788b1b4424adc94264dc56e9c1b96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0ed898e88cd410082c5c4fccb997b91', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:29.831371 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b842ec6-0052-4697-885a-5f4ceca1e84a tempest-VolumeQuotaClassesTest-1046517315 tempest-VolumeQuotaClassesTest-1046517315-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:29.831652 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b842ec6-0052-4697-885a-5f4ceca1e84a tempest-VolumeQuotaClassesTest-1046517315 tempest-VolumeQuotaClassesTest-1046517315-project-admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:29.831922 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b842ec6-0052-4697-885a-5f4ceca1e84a tempest-VolumeQuotaClassesTest-1046517315 tempest-VolumeQuotaClassesTest-1046517315-project-admin] PATH_INFO: `/v3/projects/b5ce777052ea4196aec1cc2502ebe03e` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:29.833015 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b842ec6-0052-4697-885a-5f4ceca1e84a tempest-VolumeQuotaClassesTest-1046517315 tempest-VolumeQuotaClassesTest-1046517315-project-admin] RBAC: Authorizing `identity:delete_project(project_id=b5ce777052ea4196aec1cc2502ebe03e)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:29.834908 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b842ec6-0052-4697-885a-5f4ceca1e84a tempest-VolumeQuotaClassesTest-1046517315 tempest-VolumeQuotaClassesTest-1046517315-project-admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:29.856107 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:38:29.881703 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-8b842ec6-0052-4697-885a-5f4ceca1e84a tempest-VolumeQuotaClassesTest-1046517315 tempest-VolumeQuotaClassesTest-1046517315-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5ce777052ea4196aec1cc2502ebe03e', 'request_id': 'req-8b842ec6-0052-4697-885a-5f4ceca1e84a'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:29.889179 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-8b842ec6-0052-4697-885a-5f4ceca1e84a tempest-VolumeQuotaClassesTest-1046517315 tempest-VolumeQuotaClassesTest-1046517315-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5ce777052ea4196aec1cc2502ebe03e', 'request_id': 'req-8b842ec6-0052-4697-885a-5f4ceca1e84a'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:29.898261 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-8b842ec6-0052-4697-885a-5f4ceca1e84a tempest-VolumeQuotaClassesTest-1046517315 tempest-VolumeQuotaClassesTest-1046517315-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5ce777052ea4196aec1cc2502ebe03e', 'request_id': 'req-8b842ec6-0052-4697-885a-5f4ceca1e84a'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:29.910825 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-8b842ec6-0052-4697-885a-5f4ceca1e84a tempest-VolumeQuotaClassesTest-1046517315 tempest-VolumeQuotaClassesTest-1046517315-project-admin] The token cache is being invalidate because project b5ce777052ea4196aec1cc2502ebe03e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:38:29.911111 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-8b842ec6-0052-4697-885a-5f4ceca1e84a tempest-VolumeQuotaClassesTest-1046517315 tempest-VolumeQuotaClassesTest-1046517315-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:29.911800 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-8b842ec6-0052-4697-885a-5f4ceca1e84a tempest-VolumeQuotaClassesTest-1046517315 tempest-VolumeQuotaClassesTest-1046517315-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:29.913419 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1010/2013] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:38:29 2022] DELETE /identity/v3/projects/b5ce777052ea4196aec1cc2502ebe03e => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:38:30.124079 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ace8c8-3351-4ea5-91fe-b1b63186050c tempest-VolumeQuotaClassesTest-1046517315 tempest-VolumeQuotaClassesTest-1046517315-project-admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:30.125941 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-c4ace8c8-3351-4ea5-91fe-b1b63186050c tempest-VolumeQuotaClassesTest-1046517315 tempest-VolumeQuotaClassesTest-1046517315-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:38:30.192840 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ace8c8-3351-4ea5-91fe-b1b63186050c tempest-VolumeQuotaClassesTest-1046517315 tempest-VolumeQuotaClassesTest-1046517315-project-admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:30.257672 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6244946-1e59-4914-b49c-281e37775ce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:30.258845 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6244946-1e59-4914-b49c-281e37775ce0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:30.258845 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6244946-1e59-4914-b49c-281e37775ce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:30.258845 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6244946-1e59-4914-b49c-281e37775ce0 admin admin] PATH_INFO: `/v3/users/db8f776debf54b5ca62fafe2c8277c29` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:30.260017 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6244946-1e59-4914-b49c-281e37775ce0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db8f776debf54b5ca62fafe2c8277c29)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:30.261986 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6244946-1e59-4914-b49c-281e37775ce0 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:30.310704 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:38:30.318021 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-e6244946-1e59-4914-b49c-281e37775ce0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db8f776debf54b5ca62fafe2c8277c29'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:30.326619 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-e6244946-1e59-4914-b49c-281e37775ce0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db8f776debf54b5ca62fafe2c8277c29'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:30.327435 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-e6244946-1e59-4914-b49c-281e37775ce0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db8f776debf54b5ca62fafe2c8277c29'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:30.340794 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-e6244946-1e59-4914-b49c-281e37775ce0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db8f776debf54b5ca62fafe2c8277c29'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:30.347792 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-e6244946-1e59-4914-b49c-281e37775ce0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db8f776debf54b5ca62fafe2c8277c29'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:30.354861 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1004/2014] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:38:30 2022] DELETE /identity/v3/users/db8f776debf54b5ca62fafe2c8277c29 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:38:30.359323 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dce2544-c192-4a62-9d39-a800d45a9e4d None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:30.359608 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dce2544-c192-4a62-9d39-a800d45a9e4d None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:30.359879 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dce2544-c192-4a62-9d39-a800d45a9e4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:30.378957 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-7dce2544-c192-4a62-9d39-a800d45a9e4d None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:38:30.380182 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-7dce2544-c192-4a62-9d39-a800d45a9e4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:38:30.442362 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1011/2015] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:38:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:38:30.453692 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6244946-1e59-4914-b49c-281e37775ce0 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:30.455251 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-e6244946-1e59-4914-b49c-281e37775ce0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:38:30.528000 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6244946-1e59-4914-b49c-281e37775ce0 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:30.531685 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6cec1a-9c20-412a-bc07-59b171e5c5e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:30.532286 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6cec1a-9c20-412a-bc07-59b171e5c5e1 service neutron] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:30.532562 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6cec1a-9c20-412a-bc07-59b171e5c5e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:30.532831 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6cec1a-9c20-412a-bc07-59b171e5c5e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:30.541826 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6cec1a-9c20-412a-bc07-59b171e5c5e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:30.543261 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6cec1a-9c20-412a-bc07-59b171e5c5e1 service neutron] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:30.552613 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1005/2016] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:38:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:38:30.813425 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dce2544-c192-4a62-9d39-a800d45a9e4d None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:30.815357 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-7dce2544-c192-4a62-9d39-a800d45a9e4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:38:30.830447 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dce2544-c192-4a62-9d39-a800d45a9e4d None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:30.834754 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98216da6-e7ad-466f-ab28-2ea2fc508516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:30.835492 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98216da6-e7ad-466f-ab28-2ea2fc508516 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:30.837228 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98216da6-e7ad-466f-ab28-2ea2fc508516 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:30.837516 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98216da6-e7ad-466f-ab28-2ea2fc508516 admin admin] PATH_INFO: `/v3/projects/30e029902ef7448883710930b23ef048` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:30.838703 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98216da6-e7ad-466f-ab28-2ea2fc508516 admin admin] RBAC: Authorizing `identity:delete_project(project_id=30e029902ef7448883710930b23ef048)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:30.840775 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98216da6-e7ad-466f-ab28-2ea2fc508516 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:30.886758 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-98216da6-e7ad-466f-ab28-2ea2fc508516 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30e029902ef7448883710930b23ef048', 'request_id': 'req-98216da6-e7ad-466f-ab28-2ea2fc508516'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:30.892820 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-98216da6-e7ad-466f-ab28-2ea2fc508516 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30e029902ef7448883710930b23ef048', 'request_id': 'req-98216da6-e7ad-466f-ab28-2ea2fc508516'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:30.901326 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-98216da6-e7ad-466f-ab28-2ea2fc508516 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30e029902ef7448883710930b23ef048', 'request_id': 'req-98216da6-e7ad-466f-ab28-2ea2fc508516'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:30.911518 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-98216da6-e7ad-466f-ab28-2ea2fc508516 admin admin] The token cache is being invalidate because project 30e029902ef7448883710930b23ef048 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:38:30.911804 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-98216da6-e7ad-466f-ab28-2ea2fc508516 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:30.912410 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-98216da6-e7ad-466f-ab28-2ea2fc508516 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:30.913896 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1012/2017] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:38:30 2022] DELETE /identity/v3/projects/30e029902ef7448883710930b23ef048 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:38:30.918266 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6cec1a-9c20-412a-bc07-59b171e5c5e1 service neutron] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:30.927669 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6cec1a-9c20-412a-bc07-59b171e5c5e1 service neutron] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:30.982764 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c4e3bd-1d79-49b6-81fc-69e15799fdc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:30.983428 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c4e3bd-1d79-49b6-81fc-69e15799fdc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:30.983705 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c4e3bd-1d79-49b6-81fc-69e15799fdc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:30.983974 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c4e3bd-1d79-49b6-81fc-69e15799fdc1 admin admin] PATH_INFO: `/v3/users/d11788b1b4424adc94264dc56e9c1b96` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:30.985061 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c4e3bd-1d79-49b6-81fc-69e15799fdc1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d11788b1b4424adc94264dc56e9c1b96)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:30.986986 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c4e3bd-1d79-49b6-81fc-69e15799fdc1 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:31.039286 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-85c4e3bd-1d79-49b6-81fc-69e15799fdc1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd11788b1b4424adc94264dc56e9c1b96'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:31.047790 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-85c4e3bd-1d79-49b6-81fc-69e15799fdc1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd11788b1b4424adc94264dc56e9c1b96'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:31.048457 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-85c4e3bd-1d79-49b6-81fc-69e15799fdc1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd11788b1b4424adc94264dc56e9c1b96'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:31.057598 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:38:31.070240 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-85c4e3bd-1d79-49b6-81fc-69e15799fdc1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd11788b1b4424adc94264dc56e9c1b96'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:31.079105 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-85c4e3bd-1d79-49b6-81fc-69e15799fdc1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd11788b1b4424adc94264dc56e9c1b96'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:31.086691 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1006/2018] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:38:30 2022] DELETE /identity/v3/users/d11788b1b4424adc94264dc56e9c1b96 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:38:31.474622 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98216da6-e7ad-466f-ab28-2ea2fc508516 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:31.476075 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-98216da6-e7ad-466f-ab28-2ea2fc508516 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:38:31.489431 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98216da6-e7ad-466f-ab28-2ea2fc508516 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:31.520072 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:38:31.548591 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42850abc-2e7d-4791-a20f-7608b6da7d71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:31.549151 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42850abc-2e7d-4791-a20f-7608b6da7d71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:31.549429 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42850abc-2e7d-4791-a20f-7608b6da7d71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:31.549701 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42850abc-2e7d-4791-a20f-7608b6da7d71 admin admin] PATH_INFO: `/v3/projects/f0ed898e88cd410082c5c4fccb997b91` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:31.550800 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42850abc-2e7d-4791-a20f-7608b6da7d71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0ed898e88cd410082c5c4fccb997b91)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:31.552728 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42850abc-2e7d-4791-a20f-7608b6da7d71 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:31.598248 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-42850abc-2e7d-4791-a20f-7608b6da7d71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0ed898e88cd410082c5c4fccb997b91', 'request_id': 'req-42850abc-2e7d-4791-a20f-7608b6da7d71'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:31.604178 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-42850abc-2e7d-4791-a20f-7608b6da7d71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0ed898e88cd410082c5c4fccb997b91', 'request_id': 'req-42850abc-2e7d-4791-a20f-7608b6da7d71'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:31.611833 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-42850abc-2e7d-4791-a20f-7608b6da7d71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0ed898e88cd410082c5c4fccb997b91', 'request_id': 'req-42850abc-2e7d-4791-a20f-7608b6da7d71'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:31.623421 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-42850abc-2e7d-4791-a20f-7608b6da7d71 admin admin] The token cache is being invalidate because project f0ed898e88cd410082c5c4fccb997b91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:38:31.623861 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-42850abc-2e7d-4791-a20f-7608b6da7d71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:31.624532 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-42850abc-2e7d-4791-a20f-7608b6da7d71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:31.626274 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1013/2019] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:38:31 2022] DELETE /identity/v3/projects/f0ed898e88cd410082c5c4fccb997b91 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:38:31.630994 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388305c7-6533-4a66-a3d1-872b1c495458 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:31.631551 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388305c7-6533-4a66-a3d1-872b1c495458 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:31.631551 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388305c7-6533-4a66-a3d1-872b1c495458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:31.651738 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-388305c7-6533-4a66-a3d1-872b1c495458 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:38:31.652915 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-388305c7-6533-4a66-a3d1-872b1c495458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:38:31.710445 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1007/2020] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:38:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:38:31.718755 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc2fb85-587b-4395-a762-907dcdb6708d None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:31.719038 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc2fb85-587b-4395-a762-907dcdb6708d None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:31.719361 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc2fb85-587b-4395-a762-907dcdb6708d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:31.740567 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-2bc2fb85-587b-4395-a762-907dcdb6708d None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:38:31.741751 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-2bc2fb85-587b-4395-a762-907dcdb6708d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:38:31.749766 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1014/2021] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:38:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:38:31.755339 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388305c7-6533-4a66-a3d1-872b1c495458 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:31.765193 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388305c7-6533-4a66-a3d1-872b1c495458 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:31.769935 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b081009-24e5-4f04-89c0-c2354d5e56b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:31.770525 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b081009-24e5-4f04-89c0-c2354d5e56b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:31.770799 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b081009-24e5-4f04-89c0-c2354d5e56b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:31.771064 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b081009-24e5-4f04-89c0-c2354d5e56b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:31.771736 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b081009-24e5-4f04-89c0-c2354d5e56b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:38:31.772031 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b081009-24e5-4f04-89c0-c2354d5e56b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:31.773590 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b081009-24e5-4f04-89c0-c2354d5e56b2 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:31.786028 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1008/2022] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Oct 31 12:38:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:38:31.791041 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc2fb85-587b-4395-a762-907dcdb6708d None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:31.800307 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc2fb85-587b-4395-a762-907dcdb6708d None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:31.804886 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1954368f-3451-46ca-ae04-8eb7fbcda3d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:31.805480 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1954368f-3451-46ca-ae04-8eb7fbcda3d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:31.805758 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1954368f-3451-46ca-ae04-8eb7fbcda3d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:31.806222 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1954368f-3451-46ca-ae04-8eb7fbcda3d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:31.806947 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1954368f-3451-46ca-ae04-8eb7fbcda3d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:31.808866 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1954368f-3451-46ca-ae04-8eb7fbcda3d8 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:31.835101 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1015/2023] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:38:31 2022] POST /identity/v3/projects => generated 391 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:38:31.839748 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b081009-24e5-4f04-89c0-c2354d5e56b2 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:31.849639 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b081009-24e5-4f04-89c0-c2354d5e56b2 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:31.905701 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298b9bfd-09e0-4d21-a0c8-8b3f9cc509e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:31.906305 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b9bfd-09e0-4d21-a0c8-8b3f9cc509e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:31.906587 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b9bfd-09e0-4d21-a0c8-8b3f9cc509e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:31.906860 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b9bfd-09e0-4d21-a0c8-8b3f9cc509e4 admin admin] PATH_INFO: `/v3/users` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:31.907530 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298b9bfd-09e0-4d21-a0c8-8b3f9cc509e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:31.909562 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298b9bfd-09e0-4d21-a0c8-8b3f9cc509e4 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:31.930417 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1009/2024] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:38:31 2022] POST /identity/v3/users => generated 296 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:38:31.934473 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1954368f-3451-46ca-ae04-8eb7fbcda3d8 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:31.944588 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1954368f-3451-46ca-ae04-8eb7fbcda3d8 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:31.949776 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4662e15-9a4d-490b-a5ad-b23809672497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:31.950371 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4662e15-9a4d-490b-a5ad-b23809672497 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:31.950649 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4662e15-9a4d-490b-a5ad-b23809672497 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:31.950918 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4662e15-9a4d-490b-a5ad-b23809672497 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:31.951590 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4662e15-9a4d-490b-a5ad-b23809672497 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:31.953162 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4662e15-9a4d-490b-a5ad-b23809672497 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:31.963829 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1016/2025] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:38:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:38:31.967559 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298b9bfd-09e0-4d21-a0c8-8b3f9cc509e4 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:31.977493 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298b9bfd-09e0-4d21-a0c8-8b3f9cc509e4 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:31.982328 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e56a29-857f-432e-a121-dd5da85c657f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:31.982916 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e56a29-857f-432e-a121-dd5da85c657f admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:31.983229 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e56a29-857f-432e-a121-dd5da85c657f admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:31.983501 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e56a29-857f-432e-a121-dd5da85c657f admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:31.984126 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e56a29-857f-432e-a121-dd5da85c657f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:31.985765 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e56a29-857f-432e-a121-dd5da85c657f admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:31.996250 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1010/2026] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:38:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:38:32.000324 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4662e15-9a4d-490b-a5ad-b23809672497 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:32.010475 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4662e15-9a4d-490b-a5ad-b23809672497 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:32.015017 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85506586-5a53-4e4c-96c7-494856e0a749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:32.015701 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85506586-5a53-4e4c-96c7-494856e0a749 admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:32.015988 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85506586-5a53-4e4c-96c7-494856e0a749 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:32.016263 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85506586-5a53-4e4c-96c7-494856e0a749 admin admin] PATH_INFO: `/v3/projects/2068043ed1ef4db48ec49f87bf15d054/users/18c4b4f71cc042689f18e40f85b56010/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:32.033263 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85506586-5a53-4e4c-96c7-494856e0a749 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2068043ed1ef4db48ec49f87bf15d054, user_id=18c4b4f71cc042689f18e40f85b56010, role_id=ca1a5695f8bd499aa0b8cedaeecd2cc6)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:32.035966 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85506586-5a53-4e4c-96c7-494856e0a749 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:32.049815 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1017/2027] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:38:31 2022] PUT /identity/v3/projects/2068043ed1ef4db48ec49f87bf15d054/users/18c4b4f71cc042689f18e40f85b56010/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 31 12:38:32.061754 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63922f9-7efd-4a68-a29b-51d660fe7444 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:32.061956 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63922f9-7efd-4a68-a29b-51d660fe7444 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:32.062226 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63922f9-7efd-4a68-a29b-51d660fe7444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:32.089283 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-b63922f9-7efd-4a68-a29b-51d660fe7444 None None] MFA Rules not processed for user `18c4b4f71cc042689f18e40f85b56010`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:38:32.090468 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-b63922f9-7efd-4a68-a29b-51d660fe7444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:38:32.123953 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:38:32.158034 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1011/2028] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:38:32 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:38:32.162522 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85506586-5a53-4e4c-96c7-494856e0a749 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:32.220668 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:38:32.224665 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85506586-5a53-4e4c-96c7-494856e0a749 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:32.232195 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f32862-c94c-47c1-8325-06e6a24c22a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:32.233019 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f32862-c94c-47c1-8325-06e6a24c22a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:32.233470 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f32862-c94c-47c1-8325-06e6a24c22a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:32.233785 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f32862-c94c-47c1-8325-06e6a24c22a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:32.234697 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f32862-c94c-47c1-8325-06e6a24c22a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:32.236680 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f32862-c94c-47c1-8325-06e6a24c22a1 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:32.264779 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1018/2029] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:38:32 2022] POST /identity/v3/projects => generated 391 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:38:32.268303 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63922f9-7efd-4a68-a29b-51d660fe7444 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:32.280891 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63922f9-7efd-4a68-a29b-51d660fe7444 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:32.349230 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a77da6-77c4-4bd5-9d40-3640e675488c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:32.349789 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a77da6-77c4-4bd5-9d40-3640e675488c admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:32.350068 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a77da6-77c4-4bd5-9d40-3640e675488c admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:32.350342 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a77da6-77c4-4bd5-9d40-3640e675488c admin admin] PATH_INFO: `/v3/users` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:32.351025 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a77da6-77c4-4bd5-9d40-3640e675488c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:32.353720 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a77da6-77c4-4bd5-9d40-3640e675488c admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:32.375830 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1012/2030] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:38:32 2022] POST /identity/v3/users => generated 302 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:38:32.379960 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f32862-c94c-47c1-8325-06e6a24c22a1 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:32.390522 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f32862-c94c-47c1-8325-06e6a24c22a1 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:32.395509 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56c23f9-c98b-4d46-8453-e8de1cc7231a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:32.396120 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56c23f9-c98b-4d46-8453-e8de1cc7231a admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:32.396401 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56c23f9-c98b-4d46-8453-e8de1cc7231a admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:32.396674 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56c23f9-c98b-4d46-8453-e8de1cc7231a admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:32.397349 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56c23f9-c98b-4d46-8453-e8de1cc7231a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:32.398933 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56c23f9-c98b-4d46-8453-e8de1cc7231a admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:32.411404 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1019/2031] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:38:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 31 12:38:32.416172 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a77da6-77c4-4bd5-9d40-3640e675488c admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:32.427543 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a77da6-77c4-4bd5-9d40-3640e675488c admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:32.433617 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5c980a-b027-4b74-a177-36c5294bfb00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:32.434428 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5c980a-b027-4b74-a177-36c5294bfb00 admin admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:32.435106 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5c980a-b027-4b74-a177-36c5294bfb00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:32.435573 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5c980a-b027-4b74-a177-36c5294bfb00 admin admin] PATH_INFO: `/v3/projects/b0503d511e94490ca2df90fcaceefc07/users/33d55a019f284d8c819df9f23c6e28a3/roles/176c07b4fe3b4537bdfeb4be21b379c5` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:32.453232 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5c980a-b027-4b74-a177-36c5294bfb00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0503d511e94490ca2df90fcaceefc07, user_id=33d55a019f284d8c819df9f23c6e28a3, role_id=176c07b4fe3b4537bdfeb4be21b379c5)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:32.455948 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5c980a-b027-4b74-a177-36c5294bfb00 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:32.470906 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1013/2032] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:38:32 2022] PUT /identity/v3/projects/b0503d511e94490ca2df90fcaceefc07/users/33d55a019f284d8c819df9f23c6e28a3/roles/176c07b4fe3b4537bdfeb4be21b379c5 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:38:32.474789 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56c23f9-c98b-4d46-8453-e8de1cc7231a admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:32.485917 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56c23f9-c98b-4d46-8453-e8de1cc7231a admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:32.550943 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbb665d-27ba-47cb-8811-44c20c832699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:32.551605 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbb665d-27ba-47cb-8811-44c20c832699 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:32.551911 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbb665d-27ba-47cb-8811-44c20c832699 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:32.552182 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbb665d-27ba-47cb-8811-44c20c832699 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:32.552821 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbb665d-27ba-47cb-8811-44c20c832699 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:32.554609 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbb665d-27ba-47cb-8811-44c20c832699 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:32.565753 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1020/2033] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:38:32 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 31 12:38:32.570006 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5c980a-b027-4b74-a177-36c5294bfb00 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:32.587931 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5c980a-b027-4b74-a177-36c5294bfb00 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:32.593208 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b269bf8-0947-407f-b5ee-2856ee88cff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:32.593874 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b269bf8-0947-407f-b5ee-2856ee88cff2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:32.594154 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b269bf8-0947-407f-b5ee-2856ee88cff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:32.594424 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b269bf8-0947-407f-b5ee-2856ee88cff2 admin admin] PATH_INFO: `/v3/projects/b0503d511e94490ca2df90fcaceefc07/users/33d55a019f284d8c819df9f23c6e28a3/roles/176c07b4fe3b4537bdfeb4be21b379c5` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:32.596396 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b269bf8-0947-407f-b5ee-2856ee88cff2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0503d511e94490ca2df90fcaceefc07, user_id=33d55a019f284d8c819df9f23c6e28a3, role_id=176c07b4fe3b4537bdfeb4be21b379c5)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:32.599043 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b269bf8-0947-407f-b5ee-2856ee88cff2 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:32.620761 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1014/2034] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:38:32 2022] PUT /identity/v3/projects/b0503d511e94490ca2df90fcaceefc07/users/33d55a019f284d8c819df9f23c6e28a3/roles/176c07b4fe3b4537bdfeb4be21b379c5 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:38:32.623407 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c804ae-444b-4361-ad03-a98940e1f67c None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:32.623695 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c804ae-444b-4361-ad03-a98940e1f67c None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:32.623981 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c804ae-444b-4361-ad03-a98940e1f67c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:32.666264 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-53c804ae-444b-4361-ad03-a98940e1f67c None None] MFA Rules not processed for user `33d55a019f284d8c819df9f23c6e28a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:38:32.667542 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-53c804ae-444b-4361-ad03-a98940e1f67c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:38:32.736283 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1021/2035] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:38:32 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:38:32.748247 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b269bf8-0947-407f-b5ee-2856ee88cff2 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:32.749949 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-0b269bf8-0947-407f-b5ee-2856ee88cff2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:38:32.811574 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b269bf8-0947-407f-b5ee-2856ee88cff2 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:32.815355 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04947907-5613-4e7a-ad7a-03288b54843b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d6d7ea077d49e8a66c6c6762499702', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:32.815963 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04947907-5613-4e7a-ad7a-03288b54843b service cinder] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:32.816233 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04947907-5613-4e7a-ad7a-03288b54843b service cinder] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:32.816522 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04947907-5613-4e7a-ad7a-03288b54843b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:32.826657 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04947907-5613-4e7a-ad7a-03288b54843b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:32.828100 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04947907-5613-4e7a-ad7a-03288b54843b service cinder] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:32.837618 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1015/2036] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:38:32 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:38:32.887421 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c804ae-444b-4361-ad03-a98940e1f67c None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:32.888832 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-53c804ae-444b-4361-ad03-a98940e1f67c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:38:32.901202 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c804ae-444b-4361-ad03-a98940e1f67c None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:32.904783 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52919fcd-3bbf-482f-aff8-e00a4cd6d15d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d6d7ea077d49e8a66c6c6762499702', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:32.905385 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52919fcd-3bbf-482f-aff8-e00a4cd6d15d service cinder] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:32.905659 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52919fcd-3bbf-482f-aff8-e00a4cd6d15d service cinder] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:32.905925 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52919fcd-3bbf-482f-aff8-e00a4cd6d15d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:32.956853 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:38:32.967955 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52919fcd-3bbf-482f-aff8-e00a4cd6d15d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:32.969383 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52919fcd-3bbf-482f-aff8-e00a4cd6d15d service cinder] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:32.978870 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1022/2037] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:38:32 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:38:33.556487 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04947907-5613-4e7a-ad7a-03288b54843b service cinder] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:33.557953 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-04947907-5613-4e7a-ad7a-03288b54843b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:38:33.597147 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:38:33.639354 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04947907-5613-4e7a-ad7a-03288b54843b service cinder] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:33.645023 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda5011e-c04d-4423-85d1-1d8ab3bd5875 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99889b1dab0b471d9c77887cffefc652', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:33.645626 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda5011e-c04d-4423-85d1-1d8ab3bd5875 service nova] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:33.645903 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda5011e-c04d-4423-85d1-1d8ab3bd5875 service nova] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:33.646169 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda5011e-c04d-4423-85d1-1d8ab3bd5875 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:33.647872 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-bda5011e-c04d-4423-85d1-1d8ab3bd5875 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:38:33.660283 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda5011e-c04d-4423-85d1-1d8ab3bd5875 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:33.661771 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda5011e-c04d-4423-85d1-1d8ab3bd5875 service nova] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:33.674130 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1016/2038] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:38:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:38:36.332828 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52919fcd-3bbf-482f-aff8-e00a4cd6d15d service cinder] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:36.394609 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52919fcd-3bbf-482f-aff8-e00a4cd6d15d service cinder] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:36.399130 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a372310-909d-466a-921b-eb4d16194ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:36.400371 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a372310-909d-466a-921b-eb4d16194ae4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:36.400643 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a372310-909d-466a-921b-eb4d16194ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:36.400975 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a372310-909d-466a-921b-eb4d16194ae4 admin admin] PATH_INFO: `/v3/users/18c4b4f71cc042689f18e40f85b56010` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:36.402085 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a372310-909d-466a-921b-eb4d16194ae4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=18c4b4f71cc042689f18e40f85b56010)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:36.404028 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a372310-909d-466a-921b-eb4d16194ae4 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:36.449208 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-1a372310-909d-466a-921b-eb4d16194ae4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18c4b4f71cc042689f18e40f85b56010'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:36.455941 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-1a372310-909d-466a-921b-eb4d16194ae4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18c4b4f71cc042689f18e40f85b56010'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:36.456551 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-1a372310-909d-466a-921b-eb4d16194ae4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18c4b4f71cc042689f18e40f85b56010'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:36.457207 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-1a372310-909d-466a-921b-eb4d16194ae4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18c4b4f71cc042689f18e40f85b56010'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:36.471281 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-1a372310-909d-466a-921b-eb4d16194ae4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18c4b4f71cc042689f18e40f85b56010'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:36.484587 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1023/2039] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:38:36 2022] DELETE /identity/v3/users/18c4b4f71cc042689f18e40f85b56010 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:38:36.489285 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d02a86a-9d52-49d9-8725-8c1ad5d7bf41 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:36.489810 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d02a86a-9d52-49d9-8725-8c1ad5d7bf41 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:36.489810 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d02a86a-9d52-49d9-8725-8c1ad5d7bf41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:36.509266 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-1d02a86a-9d52-49d9-8725-8c1ad5d7bf41 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:38:36.510414 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-1d02a86a-9d52-49d9-8725-8c1ad5d7bf41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:38:36.569666 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1017/2040] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:38:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:38:36.579702 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a372310-909d-466a-921b-eb4d16194ae4 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:36.581006 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-1a372310-909d-466a-921b-eb4d16194ae4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:38:36.636399 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a372310-909d-466a-921b-eb4d16194ae4 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:36.640182 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86052670-59c4-475b-a2eb-a49949a78480 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:36.640801 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86052670-59c4-475b-a2eb-a49949a78480 service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:36.641078 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86052670-59c4-475b-a2eb-a49949a78480 service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:36.641356 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86052670-59c4-475b-a2eb-a49949a78480 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:36.650775 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86052670-59c4-475b-a2eb-a49949a78480 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:36.652219 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86052670-59c4-475b-a2eb-a49949a78480 service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:36.661571 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1024/2041] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:38:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:38:37.148161 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d02a86a-9d52-49d9-8725-8c1ad5d7bf41 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:37.149951 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-1d02a86a-9d52-49d9-8725-8c1ad5d7bf41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:38:37.162147 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d02a86a-9d52-49d9-8725-8c1ad5d7bf41 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:37.168709 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa084ae8-77c0-454a-a352-c1e71c2fe9b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:37.169579 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa084ae8-77c0-454a-a352-c1e71c2fe9b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:37.169980 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa084ae8-77c0-454a-a352-c1e71c2fe9b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:37.170367 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa084ae8-77c0-454a-a352-c1e71c2fe9b5 admin admin] PATH_INFO: `/v3/projects/2068043ed1ef4db48ec49f87bf15d054` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:37.171748 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa084ae8-77c0-454a-a352-c1e71c2fe9b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2068043ed1ef4db48ec49f87bf15d054)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:37.174142 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa084ae8-77c0-454a-a352-c1e71c2fe9b5 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:37.221431 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-aa084ae8-77c0-454a-a352-c1e71c2fe9b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2068043ed1ef4db48ec49f87bf15d054', 'request_id': 'req-aa084ae8-77c0-454a-a352-c1e71c2fe9b5'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:37.226966 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-aa084ae8-77c0-454a-a352-c1e71c2fe9b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2068043ed1ef4db48ec49f87bf15d054', 'request_id': 'req-aa084ae8-77c0-454a-a352-c1e71c2fe9b5'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:37.235484 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-aa084ae8-77c0-454a-a352-c1e71c2fe9b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2068043ed1ef4db48ec49f87bf15d054', 'request_id': 'req-aa084ae8-77c0-454a-a352-c1e71c2fe9b5'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:37.247092 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-aa084ae8-77c0-454a-a352-c1e71c2fe9b5 admin admin] The token cache is being invalidate because project 2068043ed1ef4db48ec49f87bf15d054 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:38:37.247414 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-aa084ae8-77c0-454a-a352-c1e71c2fe9b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:37.248040 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-aa084ae8-77c0-454a-a352-c1e71c2fe9b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:37.249545 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1018/2042] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:38:37 2022] DELETE /identity/v3/projects/2068043ed1ef4db48ec49f87bf15d054 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:38:37.253647 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86052670-59c4-475b-a2eb-a49949a78480 service neutron] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:37.263206 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86052670-59c4-475b-a2eb-a49949a78480 service neutron] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:37.272067 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:38:37.325128 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47231dd6-17fe-4bab-8147-b0b651ee9a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:37.325922 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47231dd6-17fe-4bab-8147-b0b651ee9a75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:37.326200 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47231dd6-17fe-4bab-8147-b0b651ee9a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:37.326466 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47231dd6-17fe-4bab-8147-b0b651ee9a75 admin admin] PATH_INFO: `/v3/users/33d55a019f284d8c819df9f23c6e28a3` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:37.327846 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47231dd6-17fe-4bab-8147-b0b651ee9a75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33d55a019f284d8c819df9f23c6e28a3)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:37.329844 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47231dd6-17fe-4bab-8147-b0b651ee9a75 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:37.387375 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-47231dd6-17fe-4bab-8147-b0b651ee9a75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33d55a019f284d8c819df9f23c6e28a3'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:37.396518 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-47231dd6-17fe-4bab-8147-b0b651ee9a75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33d55a019f284d8c819df9f23c6e28a3'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:37.397202 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-47231dd6-17fe-4bab-8147-b0b651ee9a75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33d55a019f284d8c819df9f23c6e28a3'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:37.398033 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-47231dd6-17fe-4bab-8147-b0b651ee9a75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33d55a019f284d8c819df9f23c6e28a3'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:37.414420 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-47231dd6-17fe-4bab-8147-b0b651ee9a75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33d55a019f284d8c819df9f23c6e28a3'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:37.430420 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1025/2043] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:38:37 2022] DELETE /identity/v3/users/33d55a019f284d8c819df9f23c6e28a3 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:38:37.713864 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa084ae8-77c0-454a-a352-c1e71c2fe9b5 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:37.715471 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-aa084ae8-77c0-454a-a352-c1e71c2fe9b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:38:37.730673 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa084ae8-77c0-454a-a352-c1e71c2fe9b5 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:37.769604 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:38:37.792049 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b75f83-431b-4a20-8b5d-236ce9246029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:37.792773 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b75f83-431b-4a20-8b5d-236ce9246029 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:37.793055 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b75f83-431b-4a20-8b5d-236ce9246029 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:37.793327 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b75f83-431b-4a20-8b5d-236ce9246029 admin admin] PATH_INFO: `/v3/projects/b0503d511e94490ca2df90fcaceefc07` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:37.794451 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b75f83-431b-4a20-8b5d-236ce9246029 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0503d511e94490ca2df90fcaceefc07)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:37.796410 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b75f83-431b-4a20-8b5d-236ce9246029 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:37.842062 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-70b75f83-431b-4a20-8b5d-236ce9246029 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0503d511e94490ca2df90fcaceefc07', 'request_id': 'req-70b75f83-431b-4a20-8b5d-236ce9246029'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:37.847478 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-70b75f83-431b-4a20-8b5d-236ce9246029 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0503d511e94490ca2df90fcaceefc07', 'request_id': 'req-70b75f83-431b-4a20-8b5d-236ce9246029'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:37.855422 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-70b75f83-431b-4a20-8b5d-236ce9246029 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0503d511e94490ca2df90fcaceefc07', 'request_id': 'req-70b75f83-431b-4a20-8b5d-236ce9246029'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:37.866254 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-70b75f83-431b-4a20-8b5d-236ce9246029 admin admin] The token cache is being invalidate because project b0503d511e94490ca2df90fcaceefc07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:38:37.866536 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-70b75f83-431b-4a20-8b5d-236ce9246029 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:37.867146 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-70b75f83-431b-4a20-8b5d-236ce9246029 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:37.869326 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1019/2044] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:38:37 2022] DELETE /identity/v3/projects/b0503d511e94490ca2df90fcaceefc07 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 31 12:38:37.875120 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ee418c-ace8-4e6e-9af2-a282c9adef71 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:37.875481 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ee418c-ace8-4e6e-9af2-a282c9adef71 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:37.875773 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ee418c-ace8-4e6e-9af2-a282c9adef71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:37.895430 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-a5ee418c-ace8-4e6e-9af2-a282c9adef71 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:38:37.896626 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-a5ee418c-ace8-4e6e-9af2-a282c9adef71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:38:37.954737 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1026/2045] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:38:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:38:37.963528 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70133eb-7976-4853-9ed3-5490829ba3e6 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:37.963836 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70133eb-7976-4853-9ed3-5490829ba3e6 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:37.964107 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70133eb-7976-4853-9ed3-5490829ba3e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:37.984372 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-e70133eb-7976-4853-9ed3-5490829ba3e6 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:38:37.985571 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-e70133eb-7976-4853-9ed3-5490829ba3e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:38:37.994233 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1020/2046] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:38:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:38:37.998475 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ee418c-ace8-4e6e-9af2-a282c9adef71 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:38.008085 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ee418c-ace8-4e6e-9af2-a282c9adef71 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:38.012695 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a521d57e-ead3-4d8c-8e1b-f4c54b2e2f86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:38.013289 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a521d57e-ead3-4d8c-8e1b-f4c54b2e2f86 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:38.013562 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a521d57e-ead3-4d8c-8e1b-f4c54b2e2f86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:38.013829 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a521d57e-ead3-4d8c-8e1b-f4c54b2e2f86 admin admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:38.014467 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a521d57e-ead3-4d8c-8e1b-f4c54b2e2f86 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:38:38.014853 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a521d57e-ead3-4d8c-8e1b-f4c54b2e2f86 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:38.016380 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a521d57e-ead3-4d8c-8e1b-f4c54b2e2f86 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:38.031676 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1027/2047] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Oct 31 12:38:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:38:38.033235 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70133eb-7976-4853-9ed3-5490829ba3e6 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:38.043491 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70133eb-7976-4853-9ed3-5490829ba3e6 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:38.048089 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8900b98-35d7-4434-8853-4c03abc5c747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:38.048678 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8900b98-35d7-4434-8853-4c03abc5c747 admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:38.048982 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8900b98-35d7-4434-8853-4c03abc5c747 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:38.049250 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8900b98-35d7-4434-8853-4c03abc5c747 admin admin] PATH_INFO: `/v3/projects` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:38.049868 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8900b98-35d7-4434-8853-4c03abc5c747 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:38.051771 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8900b98-35d7-4434-8853-4c03abc5c747 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:38.079590 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1021/2048] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:38:38 2022] POST /identity/v3/projects => generated 379 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:38:38.083870 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a521d57e-ead3-4d8c-8e1b-f4c54b2e2f86 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:38.093696 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a521d57e-ead3-4d8c-8e1b-f4c54b2e2f86 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:38.110515 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:38:38.151289 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88f4472-270a-48f5-bcd6-f58fc81eb748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:38.151938 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88f4472-270a-48f5-bcd6-f58fc81eb748 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:38.152215 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88f4472-270a-48f5-bcd6-f58fc81eb748 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:38.152484 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88f4472-270a-48f5-bcd6-f58fc81eb748 admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:38.153111 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88f4472-270a-48f5-bcd6-f58fc81eb748 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:38.155039 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88f4472-270a-48f5-bcd6-f58fc81eb748 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:38.176714 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1028/2049] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:38:38 2022] POST /identity/v3/users => generated 290 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:38:38.181423 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8900b98-35d7-4434-8853-4c03abc5c747 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:38.191466 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8900b98-35d7-4434-8853-4c03abc5c747 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:38.196003 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0c5287-a50c-4338-b535-c92c076fdc68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:38.196589 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0c5287-a50c-4338-b535-c92c076fdc68 admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:38.196862 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0c5287-a50c-4338-b535-c92c076fdc68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:38.197130 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0c5287-a50c-4338-b535-c92c076fdc68 admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:38.197755 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0c5287-a50c-4338-b535-c92c076fdc68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:38.199396 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0c5287-a50c-4338-b535-c92c076fdc68 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:38.209130 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1022/2050] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:38:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:38:38.213760 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88f4472-270a-48f5-bcd6-f58fc81eb748 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:38.224162 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88f4472-270a-48f5-bcd6-f58fc81eb748 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:38.229046 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3beb2ba9-2fe7-4378-9fdf-14c86a1d2c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:38.229640 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beb2ba9-2fe7-4378-9fdf-14c86a1d2c8e admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:38.229918 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beb2ba9-2fe7-4378-9fdf-14c86a1d2c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:38.230187 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beb2ba9-2fe7-4378-9fdf-14c86a1d2c8e admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:38.230814 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3beb2ba9-2fe7-4378-9fdf-14c86a1d2c8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:38.232419 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3beb2ba9-2fe7-4378-9fdf-14c86a1d2c8e admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:38.243016 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1029/2051] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:38:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:38:38.247834 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0c5287-a50c-4338-b535-c92c076fdc68 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:38.257598 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0c5287-a50c-4338-b535-c92c076fdc68 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:38.262404 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee33ea4-58d8-4dfb-9b96-68c6468be17b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:38.263041 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee33ea4-58d8-4dfb-9b96-68c6468be17b admin admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:38.263340 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee33ea4-58d8-4dfb-9b96-68c6468be17b admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:38.263614 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee33ea4-58d8-4dfb-9b96-68c6468be17b admin admin] PATH_INFO: `/v3/projects/405fce5912f44248bcf210a00f2433f3/users/1708c685ecc0470191c91b1031d7665e/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:38.278736 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee33ea4-58d8-4dfb-9b96-68c6468be17b admin admin] RBAC: Authorizing `identity:create_grant(project_id=405fce5912f44248bcf210a00f2433f3, user_id=1708c685ecc0470191c91b1031d7665e, role_id=ca1a5695f8bd499aa0b8cedaeecd2cc6)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:38.281449 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee33ea4-58d8-4dfb-9b96-68c6468be17b admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:38.294671 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1023/2052] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:38:38 2022] PUT /identity/v3/projects/405fce5912f44248bcf210a00f2433f3/users/1708c685ecc0470191c91b1031d7665e/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:38:38.304719 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfcddd5-4574-411f-b863-d9672a482482 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:38.305005 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfcddd5-4574-411f-b863-d9672a482482 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:38.305327 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfcddd5-4574-411f-b863-d9672a482482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:38.333365 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-9dfcddd5-4574-411f-b863-d9672a482482 None None] MFA Rules not processed for user `1708c685ecc0470191c91b1031d7665e`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:38:38.334576 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-9dfcddd5-4574-411f-b863-d9672a482482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:38:38.408570 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1030/2053] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:38:38 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:38:38.413045 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee33ea4-58d8-4dfb-9b96-68c6468be17b admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:38.474289 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee33ea4-58d8-4dfb-9b96-68c6468be17b admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:38.479808 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76b3ffe-59cc-4025-97d7-89d0b7036bc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:38.480509 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76b3ffe-59cc-4025-97d7-89d0b7036bc0 admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:38.480791 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76b3ffe-59cc-4025-97d7-89d0b7036bc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:38.481061 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76b3ffe-59cc-4025-97d7-89d0b7036bc0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:38.481685 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76b3ffe-59cc-4025-97d7-89d0b7036bc0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:38.483682 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76b3ffe-59cc-4025-97d7-89d0b7036bc0 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:38.495018 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:38:38.512531 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1024/2054] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:38:38 2022] POST /identity/v3/projects => generated 379 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:38:38.516666 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfcddd5-4574-411f-b863-d9672a482482 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:38.528910 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfcddd5-4574-411f-b863-d9672a482482 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:38.585318 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8727567-bed6-46bb-b14f-f93391e14813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:38.585922 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8727567-bed6-46bb-b14f-f93391e14813 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:38.586202 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8727567-bed6-46bb-b14f-f93391e14813 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:38.586474 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8727567-bed6-46bb-b14f-f93391e14813 admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:38.588234 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8727567-bed6-46bb-b14f-f93391e14813 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:38.590142 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8727567-bed6-46bb-b14f-f93391e14813 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:38.611014 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1031/2055] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:38:38 2022] POST /identity/v3/users => generated 290 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:38:38.614946 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76b3ffe-59cc-4025-97d7-89d0b7036bc0 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:38.625264 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76b3ffe-59cc-4025-97d7-89d0b7036bc0 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:38.630069 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b1eb2e-3962-4258-beba-0f53f6388079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:38.630443 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b1eb2e-3962-4258-beba-0f53f6388079 admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:38.630717 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b1eb2e-3962-4258-beba-0f53f6388079 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:38.630990 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b1eb2e-3962-4258-beba-0f53f6388079 admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:38.631815 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b1eb2e-3962-4258-beba-0f53f6388079 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:38.634184 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b1eb2e-3962-4258-beba-0f53f6388079 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:38.647214 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1025/2056] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:38:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:38:38.648629 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8727567-bed6-46bb-b14f-f93391e14813 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:38.659317 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8727567-bed6-46bb-b14f-f93391e14813 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:38.663925 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dcda8c-8fbf-4c06-85d9-8bf75ae27556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:38.664546 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dcda8c-8fbf-4c06-85d9-8bf75ae27556 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:38.664825 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dcda8c-8fbf-4c06-85d9-8bf75ae27556 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:38.665088 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dcda8c-8fbf-4c06-85d9-8bf75ae27556 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:38.665780 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dcda8c-8fbf-4c06-85d9-8bf75ae27556 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:38.667372 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dcda8c-8fbf-4c06-85d9-8bf75ae27556 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:38.677573 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1032/2057] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:38:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:38:38.682491 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b1eb2e-3962-4258-beba-0f53f6388079 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:38.694403 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b1eb2e-3962-4258-beba-0f53f6388079 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:38.699950 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef770cf-d8be-4fd7-b220-0e37d2851014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:38.700649 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef770cf-d8be-4fd7-b220-0e37d2851014 admin admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:38.700883 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef770cf-d8be-4fd7-b220-0e37d2851014 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:38.701164 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef770cf-d8be-4fd7-b220-0e37d2851014 admin admin] PATH_INFO: `/v3/projects/194121844a08446b8e13668f14788e2d/users/72f58b5e8eac459e83c8d68ed609ca0b/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:38.718627 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef770cf-d8be-4fd7-b220-0e37d2851014 admin admin] RBAC: Authorizing `identity:create_grant(project_id=194121844a08446b8e13668f14788e2d, user_id=72f58b5e8eac459e83c8d68ed609ca0b, role_id=ca1a5695f8bd499aa0b8cedaeecd2cc6)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:38.721366 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef770cf-d8be-4fd7-b220-0e37d2851014 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:38.735145 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1026/2058] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:38:38 2022] PUT /identity/v3/projects/194121844a08446b8e13668f14788e2d/users/72f58b5e8eac459e83c8d68ed609ca0b/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:38:38.745636 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363dc00c-f510-4288-9af4-2de93a03dc9e None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:38.746078 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363dc00c-f510-4288-9af4-2de93a03dc9e None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:38.746355 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363dc00c-f510-4288-9af4-2de93a03dc9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:38.761461 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:38:38.778598 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-363dc00c-f510-4288-9af4-2de93a03dc9e None None] MFA Rules not processed for user `72f58b5e8eac459e83c8d68ed609ca0b`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:38:38.780014 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-363dc00c-f510-4288-9af4-2de93a03dc9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:38:38.846604 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1033/2059] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:38:38 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:38:38.851292 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef770cf-d8be-4fd7-b220-0e37d2851014 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:38.912039 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef770cf-d8be-4fd7-b220-0e37d2851014 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:38.917282 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce83af5-629d-4ad8-9372-18c5d464ee47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:38.917870 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce83af5-629d-4ad8-9372-18c5d464ee47 admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:38.918143 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce83af5-629d-4ad8-9372-18c5d464ee47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:38.918408 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce83af5-629d-4ad8-9372-18c5d464ee47 admin admin] PATH_INFO: `/v3/projects` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:38.919016 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce83af5-629d-4ad8-9372-18c5d464ee47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:38.920943 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce83af5-629d-4ad8-9372-18c5d464ee47 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:38.948096 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1027/2060] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:38:38 2022] POST /identity/v3/projects => generated 377 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:38:38.952169 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363dc00c-f510-4288-9af4-2de93a03dc9e None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:38.963373 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363dc00c-f510-4288-9af4-2de93a03dc9e None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:39.019421 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e33755-d807-4da2-8325-138c8b486f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:39.019952 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e33755-d807-4da2-8325-138c8b486f6d admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:39.020230 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e33755-d807-4da2-8325-138c8b486f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:39.020504 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e33755-d807-4da2-8325-138c8b486f6d admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:39.021168 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e33755-d807-4da2-8325-138c8b486f6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:39.023079 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e33755-d807-4da2-8325-138c8b486f6d admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:39.044563 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1034/2061] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:38:38 2022] POST /identity/v3/users => generated 295 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:38:39.048471 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce83af5-629d-4ad8-9372-18c5d464ee47 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:39.059150 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce83af5-629d-4ad8-9372-18c5d464ee47 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:39.063815 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dfc8ea-3ab0-4465-962d-101cc7c51a69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:39.064401 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dfc8ea-3ab0-4465-962d-101cc7c51a69 admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:39.064674 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dfc8ea-3ab0-4465-962d-101cc7c51a69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:39.064943 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dfc8ea-3ab0-4465-962d-101cc7c51a69 admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:39.065598 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dfc8ea-3ab0-4465-962d-101cc7c51a69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:39.067186 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dfc8ea-3ab0-4465-962d-101cc7c51a69 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:39.078160 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1028/2062] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:38:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:38:39.082193 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e33755-d807-4da2-8325-138c8b486f6d admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:39.092267 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e33755-d807-4da2-8325-138c8b486f6d admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:39.096841 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657985aa-39b5-4b47-a6de-b52528957ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:39.097476 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657985aa-39b5-4b47-a6de-b52528957ce6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:39.097751 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657985aa-39b5-4b47-a6de-b52528957ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:39.098021 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657985aa-39b5-4b47-a6de-b52528957ce6 admin admin] PATH_INFO: `/v3/projects/9642ecf470094ee2b2171270fdccbee7/users/a1ccbe7ccd774c1da8fc45e9c3ce22f7/roles/176c07b4fe3b4537bdfeb4be21b379c5` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:39.113483 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657985aa-39b5-4b47-a6de-b52528957ce6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9642ecf470094ee2b2171270fdccbee7, user_id=a1ccbe7ccd774c1da8fc45e9c3ce22f7, role_id=176c07b4fe3b4537bdfeb4be21b379c5)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:39.116190 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657985aa-39b5-4b47-a6de-b52528957ce6 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:39.128590 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1035/2063] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:38:39 2022] PUT /identity/v3/projects/9642ecf470094ee2b2171270fdccbee7/users/a1ccbe7ccd774c1da8fc45e9c3ce22f7/roles/176c07b4fe3b4537bdfeb4be21b379c5 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:38:39.132481 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dfc8ea-3ab0-4465-962d-101cc7c51a69 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:39.142863 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dfc8ea-3ab0-4465-962d-101cc7c51a69 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:39.197543 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaab493-f5df-47b1-8b68-e93356414975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:39.198126 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaab493-f5df-47b1-8b68-e93356414975 admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:39.198401 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaab493-f5df-47b1-8b68-e93356414975 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:39.198669 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaab493-f5df-47b1-8b68-e93356414975 admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:39.199319 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaab493-f5df-47b1-8b68-e93356414975 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:39.200878 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaab493-f5df-47b1-8b68-e93356414975 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:39.210156 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1029/2064] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:38:39 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:38:39.214574 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657985aa-39b5-4b47-a6de-b52528957ce6 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:39.224839 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657985aa-39b5-4b47-a6de-b52528957ce6 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:39.229825 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0b715d-8b23-40ba-88de-9299f388eec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:39.230469 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0b715d-8b23-40ba-88de-9299f388eec5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:39.230745 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0b715d-8b23-40ba-88de-9299f388eec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:39.231015 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0b715d-8b23-40ba-88de-9299f388eec5 admin admin] PATH_INFO: `/v3/projects/9642ecf470094ee2b2171270fdccbee7/users/a1ccbe7ccd774c1da8fc45e9c3ce22f7/roles/176c07b4fe3b4537bdfeb4be21b379c5` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:39.232917 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0b715d-8b23-40ba-88de-9299f388eec5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9642ecf470094ee2b2171270fdccbee7, user_id=a1ccbe7ccd774c1da8fc45e9c3ce22f7, role_id=176c07b4fe3b4537bdfeb4be21b379c5)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:39.235614 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0b715d-8b23-40ba-88de-9299f388eec5 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:39.246008 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1036/2065] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:38:39 2022] PUT /identity/v3/projects/9642ecf470094ee2b2171270fdccbee7/users/a1ccbe7ccd774c1da8fc45e9c3ce22f7/roles/176c07b4fe3b4537bdfeb4be21b379c5 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:38:39.256849 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16295bcb-8189-4042-8b30-ad3907c46264 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:39.257134 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16295bcb-8189-4042-8b30-ad3907c46264 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:39.257404 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16295bcb-8189-4042-8b30-ad3907c46264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:39.285053 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-16295bcb-8189-4042-8b30-ad3907c46264 None None] MFA Rules not processed for user `a1ccbe7ccd774c1da8fc45e9c3ce22f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:38:39.286231 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-16295bcb-8189-4042-8b30-ad3907c46264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:38:39.298641 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:38:39.354957 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1030/2066] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:38:39 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:38:39.365888 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0b715d-8b23-40ba-88de-9299f388eec5 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:39.367608 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-ca0b715d-8b23-40ba-88de-9299f388eec5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:38:39.423140 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:38:39.426983 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0b715d-8b23-40ba-88de-9299f388eec5 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:39.433216 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa93f7e6-91cf-40df-9724-ea1e1cee6b7c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d6d7ea077d49e8a66c6c6762499702', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:39.433845 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa93f7e6-91cf-40df-9724-ea1e1cee6b7c service cinder] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:39.434118 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa93f7e6-91cf-40df-9724-ea1e1cee6b7c service cinder] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:39.434383 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa93f7e6-91cf-40df-9724-ea1e1cee6b7c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:39.443963 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa93f7e6-91cf-40df-9724-ea1e1cee6b7c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:39.445418 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa93f7e6-91cf-40df-9724-ea1e1cee6b7c service cinder] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:39.455899 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1037/2067] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:38:39 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:38:39.498353 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16295bcb-8189-4042-8b30-ad3907c46264 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:39.499761 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-16295bcb-8189-4042-8b30-ad3907c46264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:38:39.511577 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16295bcb-8189-4042-8b30-ad3907c46264 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:39.515126 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460251e0-61e4-4586-b2ad-b7ef6a7061bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d6d7ea077d49e8a66c6c6762499702', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:39.515746 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460251e0-61e4-4586-b2ad-b7ef6a7061bd service cinder] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:39.516021 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460251e0-61e4-4586-b2ad-b7ef6a7061bd service cinder] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:39.516284 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460251e0-61e4-4586-b2ad-b7ef6a7061bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:39.572133 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460251e0-61e4-4586-b2ad-b7ef6a7061bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:39.573463 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460251e0-61e4-4586-b2ad-b7ef6a7061bd service cinder] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:39.582203 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1031/2068] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:38:39 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:38:42.469960 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa93f7e6-91cf-40df-9724-ea1e1cee6b7c service cinder] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:42.533313 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa93f7e6-91cf-40df-9724-ea1e1cee6b7c service cinder] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:42.537827 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47842e3c-ab0c-4ce4-bbad-08e1d1055be9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:42.538455 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47842e3c-ab0c-4ce4-bbad-08e1d1055be9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:42.538730 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47842e3c-ab0c-4ce4-bbad-08e1d1055be9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:42.538997 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47842e3c-ab0c-4ce4-bbad-08e1d1055be9 admin admin] PATH_INFO: `/v3/users/1708c685ecc0470191c91b1031d7665e` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:42.540668 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47842e3c-ab0c-4ce4-bbad-08e1d1055be9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1708c685ecc0470191c91b1031d7665e)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:42.542554 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47842e3c-ab0c-4ce4-bbad-08e1d1055be9 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:42.588662 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-47842e3c-ab0c-4ce4-bbad-08e1d1055be9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1708c685ecc0470191c91b1031d7665e'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:42.595939 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-47842e3c-ab0c-4ce4-bbad-08e1d1055be9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1708c685ecc0470191c91b1031d7665e'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:42.596550 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-47842e3c-ab0c-4ce4-bbad-08e1d1055be9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1708c685ecc0470191c91b1031d7665e'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:42.597153 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-47842e3c-ab0c-4ce4-bbad-08e1d1055be9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1708c685ecc0470191c91b1031d7665e'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:42.609757 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-47842e3c-ab0c-4ce4-bbad-08e1d1055be9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1708c685ecc0470191c91b1031d7665e'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:42.621713 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1038/2069] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:38:42 2022] DELETE /identity/v3/users/1708c685ecc0470191c91b1031d7665e => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:38:42.626181 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515c4e16-aadd-412a-9340-5c3a6c1c771a None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:42.626673 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515c4e16-aadd-412a-9340-5c3a6c1c771a None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:42.626790 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515c4e16-aadd-412a-9340-5c3a6c1c771a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:42.646709 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-515c4e16-aadd-412a-9340-5c3a6c1c771a None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:38:42.647926 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-515c4e16-aadd-412a-9340-5c3a6c1c771a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:38:42.704971 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1032/2070] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:38:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:38:42.715187 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47842e3c-ab0c-4ce4-bbad-08e1d1055be9 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:42.716598 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-47842e3c-ab0c-4ce4-bbad-08e1d1055be9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:38:42.773252 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47842e3c-ab0c-4ce4-bbad-08e1d1055be9 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:42.777559 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0637b9e4-4337-41ea-a439-29659a065623 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:42.778163 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0637b9e4-4337-41ea-a439-29659a065623 service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:42.778438 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0637b9e4-4337-41ea-a439-29659a065623 service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:42.778704 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0637b9e4-4337-41ea-a439-29659a065623 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:42.788467 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0637b9e4-4337-41ea-a439-29659a065623 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:42.790746 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0637b9e4-4337-41ea-a439-29659a065623 service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:42.800192 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1039/2071] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:38:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 31 12:38:43.066645 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515c4e16-aadd-412a-9340-5c3a6c1c771a None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:43.068195 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-515c4e16-aadd-412a-9340-5c3a6c1c771a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:38:43.080872 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515c4e16-aadd-412a-9340-5c3a6c1c771a None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:43.085348 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30461e77-469a-4425-8746-55914dbd6e79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:43.085971 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30461e77-469a-4425-8746-55914dbd6e79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:43.086261 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30461e77-469a-4425-8746-55914dbd6e79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:43.086519 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30461e77-469a-4425-8746-55914dbd6e79 admin admin] PATH_INFO: `/v3/projects/405fce5912f44248bcf210a00f2433f3` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:43.087686 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30461e77-469a-4425-8746-55914dbd6e79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=405fce5912f44248bcf210a00f2433f3)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:43.089588 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30461e77-469a-4425-8746-55914dbd6e79 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:43.135456 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:38:43.138762 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-30461e77-469a-4425-8746-55914dbd6e79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '405fce5912f44248bcf210a00f2433f3', 'request_id': 'req-30461e77-469a-4425-8746-55914dbd6e79'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:43.144993 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-30461e77-469a-4425-8746-55914dbd6e79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '405fce5912f44248bcf210a00f2433f3', 'request_id': 'req-30461e77-469a-4425-8746-55914dbd6e79'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:43.152880 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-30461e77-469a-4425-8746-55914dbd6e79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '405fce5912f44248bcf210a00f2433f3', 'request_id': 'req-30461e77-469a-4425-8746-55914dbd6e79'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:43.163839 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-30461e77-469a-4425-8746-55914dbd6e79 admin admin] The token cache is being invalidate because project 405fce5912f44248bcf210a00f2433f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:38:43.164125 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-30461e77-469a-4425-8746-55914dbd6e79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:43.165227 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-30461e77-469a-4425-8746-55914dbd6e79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:43.166710 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1033/2072] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:38:43 2022] DELETE /identity/v3/projects/405fce5912f44248bcf210a00f2433f3 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:38:43.170733 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0637b9e4-4337-41ea-a439-29659a065623 service neutron] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:43.180292 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0637b9e4-4337-41ea-a439-29659a065623 service neutron] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:43.189966 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:38:43.239440 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5816094d-f635-41bf-80c1-98064ab4d3d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:43.240047 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5816094d-f635-41bf-80c1-98064ab4d3d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:43.240321 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5816094d-f635-41bf-80c1-98064ab4d3d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:43.240585 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5816094d-f635-41bf-80c1-98064ab4d3d5 admin admin] PATH_INFO: `/v3/users/72f58b5e8eac459e83c8d68ed609ca0b` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:43.241694 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5816094d-f635-41bf-80c1-98064ab4d3d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72f58b5e8eac459e83c8d68ed609ca0b)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:43.244263 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5816094d-f635-41bf-80c1-98064ab4d3d5 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:43.291399 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-5816094d-f635-41bf-80c1-98064ab4d3d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72f58b5e8eac459e83c8d68ed609ca0b'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:43.299322 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-5816094d-f635-41bf-80c1-98064ab4d3d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72f58b5e8eac459e83c8d68ed609ca0b'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:43.299933 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-5816094d-f635-41bf-80c1-98064ab4d3d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72f58b5e8eac459e83c8d68ed609ca0b'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:43.300529 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-5816094d-f635-41bf-80c1-98064ab4d3d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72f58b5e8eac459e83c8d68ed609ca0b'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:43.313133 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-5816094d-f635-41bf-80c1-98064ab4d3d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72f58b5e8eac459e83c8d68ed609ca0b'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:43.325330 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1040/2073] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:38:43 2022] DELETE /identity/v3/users/72f58b5e8eac459e83c8d68ed609ca0b => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:38:43.607298 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30461e77-469a-4425-8746-55914dbd6e79 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:43.608762 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-30461e77-469a-4425-8746-55914dbd6e79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:38:43.622337 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30461e77-469a-4425-8746-55914dbd6e79 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:43.679999 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6c695b-32f3-4415-aea8-082479d0b8ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:43.680648 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6c695b-32f3-4415-aea8-082479d0b8ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:43.680928 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6c695b-32f3-4415-aea8-082479d0b8ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:43.681194 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6c695b-32f3-4415-aea8-082479d0b8ff admin admin] PATH_INFO: `/v3/projects/194121844a08446b8e13668f14788e2d` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:43.682287 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6c695b-32f3-4415-aea8-082479d0b8ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=194121844a08446b8e13668f14788e2d)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:43.684199 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6c695b-32f3-4415-aea8-082479d0b8ff admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:43.731346 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-3a6c695b-32f3-4415-aea8-082479d0b8ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '194121844a08446b8e13668f14788e2d', 'request_id': 'req-3a6c695b-32f3-4415-aea8-082479d0b8ff'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:43.736936 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-3a6c695b-32f3-4415-aea8-082479d0b8ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '194121844a08446b8e13668f14788e2d', 'request_id': 'req-3a6c695b-32f3-4415-aea8-082479d0b8ff'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:43.744232 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-3a6c695b-32f3-4415-aea8-082479d0b8ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '194121844a08446b8e13668f14788e2d', 'request_id': 'req-3a6c695b-32f3-4415-aea8-082479d0b8ff'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:43.755027 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-3a6c695b-32f3-4415-aea8-082479d0b8ff admin admin] The token cache is being invalidate because project 194121844a08446b8e13668f14788e2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:38:43.755343 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-3a6c695b-32f3-4415-aea8-082479d0b8ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:43.755992 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-3a6c695b-32f3-4415-aea8-082479d0b8ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:43.757485 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1034/2074] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:38:43 2022] DELETE /identity/v3/projects/194121844a08446b8e13668f14788e2d => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:38:43.761548 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5816094d-f635-41bf-80c1-98064ab4d3d5 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:43.763059 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-5816094d-f635-41bf-80c1-98064ab4d3d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:38:43.776484 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5816094d-f635-41bf-80c1-98064ab4d3d5 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:43.833299 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48abd9b-2189-4cf6-9bd8-a873c23eeafe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:43.833933 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48abd9b-2189-4cf6-9bd8-a873c23eeafe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:43.834205 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48abd9b-2189-4cf6-9bd8-a873c23eeafe admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:43.834471 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48abd9b-2189-4cf6-9bd8-a873c23eeafe admin admin] PATH_INFO: `/v3/users/a1ccbe7ccd774c1da8fc45e9c3ce22f7` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:43.835608 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48abd9b-2189-4cf6-9bd8-a873c23eeafe admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1ccbe7ccd774c1da8fc45e9c3ce22f7)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:43.837566 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48abd9b-2189-4cf6-9bd8-a873c23eeafe admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:43.874565 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:38:43.885459 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-b48abd9b-2189-4cf6-9bd8-a873c23eeafe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1ccbe7ccd774c1da8fc45e9c3ce22f7'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:43.893165 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-b48abd9b-2189-4cf6-9bd8-a873c23eeafe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1ccbe7ccd774c1da8fc45e9c3ce22f7'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:43.894110 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-b48abd9b-2189-4cf6-9bd8-a873c23eeafe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1ccbe7ccd774c1da8fc45e9c3ce22f7'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:43.894751 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-b48abd9b-2189-4cf6-9bd8-a873c23eeafe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1ccbe7ccd774c1da8fc45e9c3ce22f7'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:43.906818 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-b48abd9b-2189-4cf6-9bd8-a873c23eeafe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1ccbe7ccd774c1da8fc45e9c3ce22f7'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:43.919627 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1041/2075] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:38:43 2022] DELETE /identity/v3/users/a1ccbe7ccd774c1da8fc45e9c3ce22f7 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:38:44.190127 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6c695b-32f3-4415-aea8-082479d0b8ff admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:44.191592 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-3a6c695b-32f3-4415-aea8-082479d0b8ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:38:44.204852 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6c695b-32f3-4415-aea8-082479d0b8ff admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:44.262947 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3b4223-ab24-473c-946c-55ef102e0761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:44.263618 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3b4223-ab24-473c-946c-55ef102e0761 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:44.263895 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3b4223-ab24-473c-946c-55ef102e0761 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:44.264162 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3b4223-ab24-473c-946c-55ef102e0761 admin admin] PATH_INFO: `/v3/projects/9642ecf470094ee2b2171270fdccbee7` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:44.265266 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3b4223-ab24-473c-946c-55ef102e0761 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9642ecf470094ee2b2171270fdccbee7)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:44.267184 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3b4223-ab24-473c-946c-55ef102e0761 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:44.273408 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:38:44.313521 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-be3b4223-ab24-473c-946c-55ef102e0761 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9642ecf470094ee2b2171270fdccbee7', 'request_id': 'req-be3b4223-ab24-473c-946c-55ef102e0761'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:44.318896 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-be3b4223-ab24-473c-946c-55ef102e0761 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9642ecf470094ee2b2171270fdccbee7', 'request_id': 'req-be3b4223-ab24-473c-946c-55ef102e0761'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:44.327328 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-be3b4223-ab24-473c-946c-55ef102e0761 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9642ecf470094ee2b2171270fdccbee7', 'request_id': 'req-be3b4223-ab24-473c-946c-55ef102e0761'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:44.339529 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-be3b4223-ab24-473c-946c-55ef102e0761 admin admin] The token cache is being invalidate because project 9642ecf470094ee2b2171270fdccbee7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:38:44.339816 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-be3b4223-ab24-473c-946c-55ef102e0761 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:44.340439 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-be3b4223-ab24-473c-946c-55ef102e0761 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:44.341930 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1035/2076] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:38:44 2022] DELETE /identity/v3/projects/9642ecf470094ee2b2171270fdccbee7 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:38:44.347781 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f2442c-1964-41bd-ae59-1d50fce332d5 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:44.348063 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f2442c-1964-41bd-ae59-1d50fce332d5 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:44.348365 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f2442c-1964-41bd-ae59-1d50fce332d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:44.367711 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-a7f2442c-1964-41bd-ae59-1d50fce332d5 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:38:44.369477 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-a7f2442c-1964-41bd-ae59-1d50fce332d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:38:44.427449 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1042/2077] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:38:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:38:44.436012 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaa0f9e-10cc-40bc-874a-c887f1e86691 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:44.436299 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaa0f9e-10cc-40bc-874a-c887f1e86691 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:44.436578 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaa0f9e-10cc-40bc-874a-c887f1e86691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:44.456785 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-1eaa0f9e-10cc-40bc-874a-c887f1e86691 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:38:44.457992 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-1eaa0f9e-10cc-40bc-874a-c887f1e86691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:38:44.466073 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1036/2078] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:38:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:38:44.470228 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f2442c-1964-41bd-ae59-1d50fce332d5 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:44.480072 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f2442c-1964-41bd-ae59-1d50fce332d5 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:44.484544 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cd3a76-2a8f-4820-9830-4f27d9dea557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:44.485136 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cd3a76-2a8f-4820-9830-4f27d9dea557 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:44.485411 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cd3a76-2a8f-4820-9830-4f27d9dea557 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:44.485676 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cd3a76-2a8f-4820-9830-4f27d9dea557 admin admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:44.486308 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cd3a76-2a8f-4820-9830-4f27d9dea557 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:38:44.486605 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cd3a76-2a8f-4820-9830-4f27d9dea557 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:44.488231 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cd3a76-2a8f-4820-9830-4f27d9dea557 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:44.500125 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1043/2079] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Oct 31 12:38:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:38:44.504363 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaa0f9e-10cc-40bc-874a-c887f1e86691 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:44.514464 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaa0f9e-10cc-40bc-874a-c887f1e86691 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:44.519962 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258d4f46-d250-4621-8d87-490cf986d9e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:44.520556 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258d4f46-d250-4621-8d87-490cf986d9e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:44.520833 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258d4f46-d250-4621-8d87-490cf986d9e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:44.521106 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258d4f46-d250-4621-8d87-490cf986d9e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:44.521738 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258d4f46-d250-4621-8d87-490cf986d9e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:44.523670 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258d4f46-d250-4621-8d87-490cf986d9e5 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:44.551241 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1037/2080] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:38:44 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:38:44.556065 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cd3a76-2a8f-4820-9830-4f27d9dea557 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:44.565776 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cd3a76-2a8f-4820-9830-4f27d9dea557 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:44.627806 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de8e6a1-29e9-4fc2-abb8-b2a26cc6f74b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:44.628462 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de8e6a1-29e9-4fc2-abb8-b2a26cc6f74b admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:44.628774 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de8e6a1-29e9-4fc2-abb8-b2a26cc6f74b admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:44.629046 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de8e6a1-29e9-4fc2-abb8-b2a26cc6f74b admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:44.629675 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de8e6a1-29e9-4fc2-abb8-b2a26cc6f74b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:44.631650 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de8e6a1-29e9-4fc2-abb8-b2a26cc6f74b admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:44.658606 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1044/2081] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:38:44 2022] POST /identity/v3/users => generated 293 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:38:44.659783 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258d4f46-d250-4621-8d87-490cf986d9e5 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:44.673603 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258d4f46-d250-4621-8d87-490cf986d9e5 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:44.678324 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4b52b0-51dc-48af-b992-0cb81dc05081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:44.678858 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4b52b0-51dc-48af-b992-0cb81dc05081 admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:44.679136 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4b52b0-51dc-48af-b992-0cb81dc05081 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:44.679432 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4b52b0-51dc-48af-b992-0cb81dc05081 admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:44.680068 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4b52b0-51dc-48af-b992-0cb81dc05081 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:44.682252 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4b52b0-51dc-48af-b992-0cb81dc05081 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:44.695581 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1038/2082] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:38:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 31 12:38:44.697040 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de8e6a1-29e9-4fc2-abb8-b2a26cc6f74b admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:44.707169 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de8e6a1-29e9-4fc2-abb8-b2a26cc6f74b admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:44.712038 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2209c28a-0751-4705-92a3-1778becb9486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:44.712643 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2209c28a-0751-4705-92a3-1778becb9486 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:44.712921 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2209c28a-0751-4705-92a3-1778becb9486 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:44.713191 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2209c28a-0751-4705-92a3-1778becb9486 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:44.713856 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2209c28a-0751-4705-92a3-1778becb9486 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:44.715467 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2209c28a-0751-4705-92a3-1778becb9486 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:44.727077 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1045/2083] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:38:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:38:44.731925 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4b52b0-51dc-48af-b992-0cb81dc05081 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:44.742300 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4b52b0-51dc-48af-b992-0cb81dc05081 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:44.747434 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3dbb71-9567-4d02-82ed-e797793b150c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:44.748168 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3dbb71-9567-4d02-82ed-e797793b150c admin admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:44.748550 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3dbb71-9567-4d02-82ed-e797793b150c admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:44.748887 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3dbb71-9567-4d02-82ed-e797793b150c admin admin] PATH_INFO: `/v3/projects/ba6e5950e3774c118adc1da78695fbd1/users/ecddc27110ea4e7383748a1b19b86f2f/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:44.764375 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3dbb71-9567-4d02-82ed-e797793b150c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba6e5950e3774c118adc1da78695fbd1, user_id=ecddc27110ea4e7383748a1b19b86f2f, role_id=ca1a5695f8bd499aa0b8cedaeecd2cc6)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:44.767015 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3dbb71-9567-4d02-82ed-e797793b150c admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:44.780544 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1039/2084] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:38:44 2022] PUT /identity/v3/projects/ba6e5950e3774c118adc1da78695fbd1/users/ecddc27110ea4e7383748a1b19b86f2f/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 31 12:38:44.850084 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f1828f-9a21-4377-b583-30bc8a36ae6c None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:44.850607 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f1828f-9a21-4377-b583-30bc8a36ae6c None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:44.850701 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f1828f-9a21-4377-b583-30bc8a36ae6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:44.877987 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-66f1828f-9a21-4377-b583-30bc8a36ae6c None None] MFA Rules not processed for user `ecddc27110ea4e7383748a1b19b86f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:38:44.879220 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-66f1828f-9a21-4377-b583-30bc8a36ae6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:38:44.884866 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:38:44.944637 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1046/2085] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:38:44 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:38:44.949808 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3dbb71-9567-4d02-82ed-e797793b150c admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:45.011315 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3dbb71-9567-4d02-82ed-e797793b150c admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:45.015920 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b68c347-c993-4999-8111-8248055f2c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:45.016514 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b68c347-c993-4999-8111-8248055f2c8b admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:45.016792 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b68c347-c993-4999-8111-8248055f2c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:45.017061 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b68c347-c993-4999-8111-8248055f2c8b admin admin] PATH_INFO: `/v3/projects` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:45.017680 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b68c347-c993-4999-8111-8248055f2c8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:45.019856 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b68c347-c993-4999-8111-8248055f2c8b admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:45.047404 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1040/2086] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:38:44 2022] POST /identity/v3/projects => generated 385 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:38:45.051424 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f1828f-9a21-4377-b583-30bc8a36ae6c None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:45.062764 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f1828f-9a21-4377-b583-30bc8a36ae6c None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:45.117667 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6355c8-bf35-4b68-a138-0d01af86f0ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:45.118751 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6355c8-bf35-4b68-a138-0d01af86f0ae admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:45.119028 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6355c8-bf35-4b68-a138-0d01af86f0ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:45.119328 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6355c8-bf35-4b68-a138-0d01af86f0ae admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:45.119963 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6355c8-bf35-4b68-a138-0d01af86f0ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:45.121856 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6355c8-bf35-4b68-a138-0d01af86f0ae admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:45.142814 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1047/2087] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:38:45 2022] POST /identity/v3/users => generated 299 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:38:45.147046 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b68c347-c993-4999-8111-8248055f2c8b admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:45.157682 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b68c347-c993-4999-8111-8248055f2c8b admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:45.162301 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa13a11e-5ae5-46f1-b635-43b6780b09b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:45.162900 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa13a11e-5ae5-46f1-b635-43b6780b09b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:45.163203 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa13a11e-5ae5-46f1-b635-43b6780b09b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:45.163486 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa13a11e-5ae5-46f1-b635-43b6780b09b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:45.164120 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa13a11e-5ae5-46f1-b635-43b6780b09b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:45.165779 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa13a11e-5ae5-46f1-b635-43b6780b09b1 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:45.176235 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1041/2088] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:38:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:38:45.180232 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6355c8-bf35-4b68-a138-0d01af86f0ae admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:45.190562 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6355c8-bf35-4b68-a138-0d01af86f0ae admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:45.195193 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459ab3b6-79c1-40f6-b137-534105a42e11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:45.195810 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459ab3b6-79c1-40f6-b137-534105a42e11 admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:45.196634 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459ab3b6-79c1-40f6-b137-534105a42e11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:45.196907 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459ab3b6-79c1-40f6-b137-534105a42e11 admin admin] PATH_INFO: `/v3/projects/6baca81a74744ff3943343468193098e/users/306e2d82df334a37b3a79e4a0590ddf2/roles/176c07b4fe3b4537bdfeb4be21b379c5` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:45.211226 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459ab3b6-79c1-40f6-b137-534105a42e11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6baca81a74744ff3943343468193098e, user_id=306e2d82df334a37b3a79e4a0590ddf2, role_id=176c07b4fe3b4537bdfeb4be21b379c5)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:45.213926 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459ab3b6-79c1-40f6-b137-534105a42e11 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:45.226939 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1048/2089] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:38:45 2022] PUT /identity/v3/projects/6baca81a74744ff3943343468193098e/users/306e2d82df334a37b3a79e4a0590ddf2/roles/176c07b4fe3b4537bdfeb4be21b379c5 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:38:45.231075 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa13a11e-5ae5-46f1-b635-43b6780b09b1 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:45.237421 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:38:45.241753 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa13a11e-5ae5-46f1-b635-43b6780b09b1 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:45.299796 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64426091-68af-42ec-ac4c-750589c8deba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:45.300389 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64426091-68af-42ec-ac4c-750589c8deba admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:45.300664 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64426091-68af-42ec-ac4c-750589c8deba admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:45.300934 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64426091-68af-42ec-ac4c-750589c8deba admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:45.301557 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64426091-68af-42ec-ac4c-750589c8deba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:45.303273 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64426091-68af-42ec-ac4c-750589c8deba admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:45.312829 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1042/2090] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:38:45 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:38:45.316965 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459ab3b6-79c1-40f6-b137-534105a42e11 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:45.327684 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459ab3b6-79c1-40f6-b137-534105a42e11 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:45.332359 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34c5493-d2a0-4d06-a14f-79d9247a2494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:45.333008 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34c5493-d2a0-4d06-a14f-79d9247a2494 admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:45.333285 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34c5493-d2a0-4d06-a14f-79d9247a2494 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:45.333556 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34c5493-d2a0-4d06-a14f-79d9247a2494 admin admin] PATH_INFO: `/v3/projects/6baca81a74744ff3943343468193098e/users/306e2d82df334a37b3a79e4a0590ddf2/roles/176c07b4fe3b4537bdfeb4be21b379c5` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:45.335483 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34c5493-d2a0-4d06-a14f-79d9247a2494 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6baca81a74744ff3943343468193098e, user_id=306e2d82df334a37b3a79e4a0590ddf2, role_id=176c07b4fe3b4537bdfeb4be21b379c5)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:45.338304 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34c5493-d2a0-4d06-a14f-79d9247a2494 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:45.348340 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1049/2091] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:38:45 2022] PUT /identity/v3/projects/6baca81a74744ff3943343468193098e/users/306e2d82df334a37b3a79e4a0590ddf2/roles/176c07b4fe3b4537bdfeb4be21b379c5 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:38:45.358882 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922d11f3-93ca-47eb-b083-b70d9710ab8e None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:45.359301 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922d11f3-93ca-47eb-b083-b70d9710ab8e None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:45.359582 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922d11f3-93ca-47eb-b083-b70d9710ab8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:45.387872 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-922d11f3-93ca-47eb-b083-b70d9710ab8e None None] MFA Rules not processed for user `306e2d82df334a37b3a79e4a0590ddf2`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:38:45.389078 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-922d11f3-93ca-47eb-b083-b70d9710ab8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:38:45.457340 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1043/2092] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:38:45 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:38:45.468105 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34c5493-d2a0-4d06-a14f-79d9247a2494 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:45.469779 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-c34c5493-d2a0-4d06-a14f-79d9247a2494 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:38:45.527692 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34c5493-d2a0-4d06-a14f-79d9247a2494 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:45.531214 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3ce707-07cb-4d2e-873e-9e4a1ea3f551 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d6d7ea077d49e8a66c6c6762499702', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:45.531890 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3ce707-07cb-4d2e-873e-9e4a1ea3f551 service cinder] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:45.532161 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3ce707-07cb-4d2e-873e-9e4a1ea3f551 service cinder] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:45.532426 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3ce707-07cb-4d2e-873e-9e4a1ea3f551 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:45.544405 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:38:45.589516 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3ce707-07cb-4d2e-873e-9e4a1ea3f551 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:45.590964 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3ce707-07cb-4d2e-873e-9e4a1ea3f551 service cinder] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:45.599702 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1050/2093] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:38:45 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:38:49.165077 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e69ad5-a0f2-410a-b8b3-1d71c5896fb4 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:49.165525 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e69ad5-a0f2-410a-b8b3-1d71c5896fb4 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:49.165636 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e69ad5-a0f2-410a-b8b3-1d71c5896fb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:49.185775 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-c8e69ad5-a0f2-410a-b8b3-1d71c5896fb4 None None] MFA Rules not processed for user `bf4faacf7df248d28f32d7014dadcc07`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:38:49.186960 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-c8e69ad5-a0f2-410a-b8b3-1d71c5896fb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:38:49.241843 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1044/2094] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Oct 31 12:38:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:38:49.288590 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3ce707-07cb-4d2e-873e-9e4a1ea3f551 service cinder] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:49.289981 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-de3ce707-07cb-4d2e-873e-9e4a1ea3f551 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:38:49.390631 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3ce707-07cb-4d2e-873e-9e4a1ea3f551 service cinder] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:49.395924 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5bf949-2ed8-4768-b62f-2b350ed29b15 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2254a5786c249b4aa79244f7198ae65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:49.396535 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5bf949-2ed8-4768-b62f-2b350ed29b15 service swift] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:49.396815 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5bf949-2ed8-4768-b62f-2b350ed29b15 service swift] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:49.397088 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5bf949-2ed8-4768-b62f-2b350ed29b15 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:49.405623 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5bf949-2ed8-4768-b62f-2b350ed29b15 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:49.407211 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5bf949-2ed8-4768-b62f-2b350ed29b15 service swift] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:49.416414 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1051/2095] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Oct 31 12:38:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 31 12:38:49.868612 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e69ad5-a0f2-410a-b8b3-1d71c5896fb4 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:49.870717 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-c8e69ad5-a0f2-410a-b8b3-1d71c5896fb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:38:49.882636 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e69ad5-a0f2-410a-b8b3-1d71c5896fb4 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:49.890199 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebd52f7-717f-4a69-a2e4-8717e2b3e805 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d6d7ea077d49e8a66c6c6762499702', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:49.890982 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebd52f7-717f-4a69-a2e4-8717e2b3e805 service cinder] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:49.891308 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebd52f7-717f-4a69-a2e4-8717e2b3e805 service cinder] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:49.891589 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebd52f7-717f-4a69-a2e4-8717e2b3e805 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:49.902559 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebd52f7-717f-4a69-a2e4-8717e2b3e805 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:49.904028 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebd52f7-717f-4a69-a2e4-8717e2b3e805 service cinder] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:49.914730 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1045/2096] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:38:49 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:38:50.367921 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5bf949-2ed8-4768-b62f-2b350ed29b15 service swift] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:50.370303 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-8a5bf949-2ed8-4768-b62f-2b350ed29b15 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:38:50.477906 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5bf949-2ed8-4768-b62f-2b350ed29b15 service swift] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:50.483487 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ca1dac-12cb-4138-8970-4dea78b574f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99889b1dab0b471d9c77887cffefc652', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:50.484097 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ca1dac-12cb-4138-8970-4dea78b574f0 service nova] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:50.484370 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ca1dac-12cb-4138-8970-4dea78b574f0 service nova] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:50.484636 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ca1dac-12cb-4138-8970-4dea78b574f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:50.486334 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-82ca1dac-12cb-4138-8970-4dea78b574f0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:38:50.500997 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ca1dac-12cb-4138-8970-4dea78b574f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:50.502601 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ca1dac-12cb-4138-8970-4dea78b574f0 service nova] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:50.518029 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1052/2097] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:38:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:38:55.261301 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebd52f7-717f-4a69-a2e4-8717e2b3e805 service cinder] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:55.282097 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:38:55.341517 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebd52f7-717f-4a69-a2e4-8717e2b3e805 service cinder] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:55.345568 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba84fc2-9ff1-468b-8b07-aaf776c5d9c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:55.346234 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba84fc2-9ff1-468b-8b07-aaf776c5d9c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:55.346510 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba84fc2-9ff1-468b-8b07-aaf776c5d9c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:55.346776 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba84fc2-9ff1-468b-8b07-aaf776c5d9c0 admin admin] PATH_INFO: `/v3/users/ecddc27110ea4e7383748a1b19b86f2f` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:55.348270 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba84fc2-9ff1-468b-8b07-aaf776c5d9c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ecddc27110ea4e7383748a1b19b86f2f)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:55.350172 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba84fc2-9ff1-468b-8b07-aaf776c5d9c0 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:55.406027 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-dba84fc2-9ff1-468b-8b07-aaf776c5d9c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ecddc27110ea4e7383748a1b19b86f2f'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:55.415613 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-dba84fc2-9ff1-468b-8b07-aaf776c5d9c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ecddc27110ea4e7383748a1b19b86f2f'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:55.416667 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-dba84fc2-9ff1-468b-8b07-aaf776c5d9c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ecddc27110ea4e7383748a1b19b86f2f'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:55.432105 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-dba84fc2-9ff1-468b-8b07-aaf776c5d9c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ecddc27110ea4e7383748a1b19b86f2f'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:55.440405 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-dba84fc2-9ff1-468b-8b07-aaf776c5d9c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ecddc27110ea4e7383748a1b19b86f2f'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:55.453248 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1046/2098] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:38:55 2022] DELETE /identity/v3/users/ecddc27110ea4e7383748a1b19b86f2f => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:38:55.458091 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7625230-4550-4338-8ff7-16cb2fd078d8 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:55.458492 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7625230-4550-4338-8ff7-16cb2fd078d8 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:55.458838 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7625230-4550-4338-8ff7-16cb2fd078d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:55.480819 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-c7625230-4550-4338-8ff7-16cb2fd078d8 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:38:55.482060 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-c7625230-4550-4338-8ff7-16cb2fd078d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:38:55.510835 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:38:55.554036 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1053/2099] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:38:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:38:55.565184 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba84fc2-9ff1-468b-8b07-aaf776c5d9c0 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:55.566784 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-dba84fc2-9ff1-468b-8b07-aaf776c5d9c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:38:55.663010 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba84fc2-9ff1-468b-8b07-aaf776c5d9c0 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:55.667850 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ae7721-fbc0-4edf-8f3e-aa860bb97854 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:55.668594 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ae7721-fbc0-4edf-8f3e-aa860bb97854 service neutron] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:55.668941 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ae7721-fbc0-4edf-8f3e-aa860bb97854 service neutron] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:55.669281 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ae7721-fbc0-4edf-8f3e-aa860bb97854 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:55.679886 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ae7721-fbc0-4edf-8f3e-aa860bb97854 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:55.682199 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ae7721-fbc0-4edf-8f3e-aa860bb97854 service neutron] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:55.693619 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1047/2100] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:38:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:38:55.773214 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1806ec6b-79f0-41bb-b7fb-bd6034c86494 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:55.773557 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1806ec6b-79f0-41bb-b7fb-bd6034c86494 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:55.773750 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1806ec6b-79f0-41bb-b7fb-bd6034c86494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:55.808896 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-1806ec6b-79f0-41bb-b7fb-bd6034c86494 None None] MFA Rules not processed for user `49b76d81ee9446cf9014a2e695ef03e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:38:55.810260 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-1806ec6b-79f0-41bb-b7fb-bd6034c86494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:38:55.896397 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1054/2101] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:38:55 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:38:56.309574 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ae7721-fbc0-4edf-8f3e-aa860bb97854 service neutron] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:56.311589 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-09ae7721-fbc0-4edf-8f3e-aa860bb97854 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:38:56.336675 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ae7721-fbc0-4edf-8f3e-aa860bb97854 service neutron] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:56.344064 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5fcccb-7ce0-4f8c-9fc5-b98f97498746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:56.345031 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5fcccb-7ce0-4f8c-9fc5-b98f97498746 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:56.345503 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5fcccb-7ce0-4f8c-9fc5-b98f97498746 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:56.345977 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5fcccb-7ce0-4f8c-9fc5-b98f97498746 admin admin] PATH_INFO: `/v3/projects/ba6e5950e3774c118adc1da78695fbd1` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:56.347360 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5fcccb-7ce0-4f8c-9fc5-b98f97498746 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba6e5950e3774c118adc1da78695fbd1)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:56.351024 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5fcccb-7ce0-4f8c-9fc5-b98f97498746 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:56.416511 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:38:56.421174 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1806ec6b-79f0-41bb-b7fb-bd6034c86494 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:56.433469 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1806ec6b-79f0-41bb-b7fb-bd6034c86494 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:56.436430 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-5b5fcccb-7ce0-4f8c-9fc5-b98f97498746 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba6e5950e3774c118adc1da78695fbd1', 'request_id': 'req-5b5fcccb-7ce0-4f8c-9fc5-b98f97498746'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:56.444350 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-5b5fcccb-7ce0-4f8c-9fc5-b98f97498746 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba6e5950e3774c118adc1da78695fbd1', 'request_id': 'req-5b5fcccb-7ce0-4f8c-9fc5-b98f97498746'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:56.457857 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-5b5fcccb-7ce0-4f8c-9fc5-b98f97498746 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba6e5950e3774c118adc1da78695fbd1', 'request_id': 'req-5b5fcccb-7ce0-4f8c-9fc5-b98f97498746'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:56.472890 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-5b5fcccb-7ce0-4f8c-9fc5-b98f97498746 admin admin] The token cache is being invalidate because project ba6e5950e3774c118adc1da78695fbd1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:38:56.473441 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-5b5fcccb-7ce0-4f8c-9fc5-b98f97498746 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:56.474257 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-5b5fcccb-7ce0-4f8c-9fc5-b98f97498746 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:56.475781 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1048/2102] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:38:56 2022] DELETE /identity/v3/projects/ba6e5950e3774c118adc1da78695fbd1 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:38:56.481421 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5fcccb-7ce0-4f8c-9fc5-b98f97498746 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:56.482154 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-5b5fcccb-7ce0-4f8c-9fc5-b98f97498746 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:38:56.498238 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e6a9e6-dbcf-4cdb-a920-65b08bfbe723 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:56.498843 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e6a9e6-dbcf-4cdb-a920-65b08bfbe723 service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:56.499121 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e6a9e6-dbcf-4cdb-a920-65b08bfbe723 service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:56.499425 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e6a9e6-dbcf-4cdb-a920-65b08bfbe723 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:56.501151 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-22e6a9e6-dbcf-4cdb-a920-65b08bfbe723 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:38:56.511129 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e6a9e6-dbcf-4cdb-a920-65b08bfbe723 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:56.512582 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e6a9e6-dbcf-4cdb-a920-65b08bfbe723 service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:56.530096 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1055/2103] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:38:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 31 12:38:56.575478 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5fcccb-7ce0-4f8c-9fc5-b98f97498746 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:56.585366 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fcdccd-af7d-42fc-8edf-c6f1c30126fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:56.586160 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fcdccd-af7d-42fc-8edf-c6f1c30126fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:56.586528 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fcdccd-af7d-42fc-8edf-c6f1c30126fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:56.586898 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fcdccd-af7d-42fc-8edf-c6f1c30126fe admin admin] PATH_INFO: `/v3/users/306e2d82df334a37b3a79e4a0590ddf2` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:56.588909 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fcdccd-af7d-42fc-8edf-c6f1c30126fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=306e2d82df334a37b3a79e4a0590ddf2)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:56.590938 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fcdccd-af7d-42fc-8edf-c6f1c30126fe admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:56.651721 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-09fcdccd-af7d-42fc-8edf-c6f1c30126fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '306e2d82df334a37b3a79e4a0590ddf2'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:56.667214 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-09fcdccd-af7d-42fc-8edf-c6f1c30126fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '306e2d82df334a37b3a79e4a0590ddf2'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:56.668277 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-09fcdccd-af7d-42fc-8edf-c6f1c30126fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '306e2d82df334a37b3a79e4a0590ddf2'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:56.686963 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-09fcdccd-af7d-42fc-8edf-c6f1c30126fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '306e2d82df334a37b3a79e4a0590ddf2'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:56.700808 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-09fcdccd-af7d-42fc-8edf-c6f1c30126fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '306e2d82df334a37b3a79e4a0590ddf2'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:56.719295 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1049/2104] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:38:56 2022] DELETE /identity/v3/users/306e2d82df334a37b3a79e4a0590ddf2 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:38:57.079400 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e6a9e6-dbcf-4cdb-a920-65b08bfbe723 service neutron] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:57.106252 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e6a9e6-dbcf-4cdb-a920-65b08bfbe723 service neutron] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:57.216413 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac15ec3f-fc80-4bd3-88a0-9ffc348a27fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:57.217281 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac15ec3f-fc80-4bd3-88a0-9ffc348a27fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:57.217629 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac15ec3f-fc80-4bd3-88a0-9ffc348a27fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:57.217973 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac15ec3f-fc80-4bd3-88a0-9ffc348a27fa admin admin] PATH_INFO: `/v3/projects/6baca81a74744ff3943343468193098e` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:57.219235 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac15ec3f-fc80-4bd3-88a0-9ffc348a27fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=6baca81a74744ff3943343468193098e)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:57.221189 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac15ec3f-fc80-4bd3-88a0-9ffc348a27fa admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:57.249357 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:38:57.297649 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-ac15ec3f-fc80-4bd3-88a0-9ffc348a27fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6baca81a74744ff3943343468193098e', 'request_id': 'req-ac15ec3f-fc80-4bd3-88a0-9ffc348a27fa'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:57.309780 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-ac15ec3f-fc80-4bd3-88a0-9ffc348a27fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6baca81a74744ff3943343468193098e', 'request_id': 'req-ac15ec3f-fc80-4bd3-88a0-9ffc348a27fa'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:57.325867 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-ac15ec3f-fc80-4bd3-88a0-9ffc348a27fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6baca81a74744ff3943343468193098e', 'request_id': 'req-ac15ec3f-fc80-4bd3-88a0-9ffc348a27fa'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:57.351400 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-ac15ec3f-fc80-4bd3-88a0-9ffc348a27fa admin admin] The token cache is being invalidate because project 6baca81a74744ff3943343468193098e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:38:57.351400 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-ac15ec3f-fc80-4bd3-88a0-9ffc348a27fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:57.351875 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-ac15ec3f-fc80-4bd3-88a0-9ffc348a27fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:38:57.354373 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1056/2105] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:38:57 2022] DELETE /identity/v3/projects/6baca81a74744ff3943343468193098e => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:38:57.360818 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38980c7e-d900-4886-a049-60ccd02c0dff None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:57.364646 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38980c7e-d900-4886-a049-60ccd02c0dff None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:57.365049 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38980c7e-d900-4886-a049-60ccd02c0dff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:57.406702 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-38980c7e-d900-4886-a049-60ccd02c0dff None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:38:57.408020 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-38980c7e-d900-4886-a049-60ccd02c0dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:38:57.456260 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:38:57.485720 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1050/2106] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:38:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:38:57.495678 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b38b8e4-f1bb-4db2-b453-e38783d4ba72 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:57.496226 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b38b8e4-f1bb-4db2-b453-e38783d4ba72 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:57.496647 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b38b8e4-f1bb-4db2-b453-e38783d4ba72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:57.522873 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-2b38b8e4-f1bb-4db2-b453-e38783d4ba72 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:38:57.524731 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-2b38b8e4-f1bb-4db2-b453-e38783d4ba72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:38:57.540158 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1057/2107] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:38:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:38:57.544860 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38980c7e-d900-4886-a049-60ccd02c0dff None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:57.559032 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38980c7e-d900-4886-a049-60ccd02c0dff None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:57.564919 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba2f51a-d164-4029-9d03-81a91aa91868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:57.565845 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba2f51a-d164-4029-9d03-81a91aa91868 admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:57.566133 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba2f51a-d164-4029-9d03-81a91aa91868 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:57.566409 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba2f51a-d164-4029-9d03-81a91aa91868 admin admin] PATH_INFO: `/v3/domains` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:57.567059 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba2f51a-d164-4029-9d03-81a91aa91868 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:38:57.567401 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba2f51a-d164-4029-9d03-81a91aa91868 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:57.569199 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba2f51a-d164-4029-9d03-81a91aa91868 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:57.587893 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1051/2108] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Oct 31 12:38:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:38:57.592485 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b38b8e4-f1bb-4db2-b453-e38783d4ba72 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:57.609235 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b38b8e4-f1bb-4db2-b453-e38783d4ba72 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:57.616119 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e696a4c-d5d6-4e0d-abae-015466c37063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:57.620966 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e696a4c-d5d6-4e0d-abae-015466c37063 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:57.623395 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e696a4c-d5d6-4e0d-abae-015466c37063 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:57.623565 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e696a4c-d5d6-4e0d-abae-015466c37063 admin admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:57.624297 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e696a4c-d5d6-4e0d-abae-015466c37063 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:57.627284 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e696a4c-d5d6-4e0d-abae-015466c37063 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:57.666950 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1058/2109] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:38:57 2022] POST /identity/v3/projects => generated 377 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:38:57.668336 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba2f51a-d164-4029-9d03-81a91aa91868 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:57.679666 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba2f51a-d164-4029-9d03-81a91aa91868 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:57.809203 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a5ccb8-05fb-440f-b929-7d3c9954c970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:57.809203 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a5ccb8-05fb-440f-b929-7d3c9954c970 admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:57.809563 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a5ccb8-05fb-440f-b929-7d3c9954c970 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:57.809855 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a5ccb8-05fb-440f-b929-7d3c9954c970 admin admin] PATH_INFO: `/v3/users` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:57.810834 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a5ccb8-05fb-440f-b929-7d3c9954c970 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:57.823232 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a5ccb8-05fb-440f-b929-7d3c9954c970 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:57.851295 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1052/2110] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:38:57 2022] POST /identity/v3/users => generated 289 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:38:57.856112 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e696a4c-d5d6-4e0d-abae-015466c37063 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:57.868657 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e696a4c-d5d6-4e0d-abae-015466c37063 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:57.876333 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea498962-3413-4944-8b5e-58cde981cf99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:57.877113 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea498962-3413-4944-8b5e-58cde981cf99 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:57.877497 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea498962-3413-4944-8b5e-58cde981cf99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:57.877858 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea498962-3413-4944-8b5e-58cde981cf99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:57.892096 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea498962-3413-4944-8b5e-58cde981cf99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:57.901160 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea498962-3413-4944-8b5e-58cde981cf99 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:57.922842 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1059/2111] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:38:57 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:38:57.923966 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a5ccb8-05fb-440f-b929-7d3c9954c970 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:57.936556 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a5ccb8-05fb-440f-b929-7d3c9954c970 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:57.941849 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6bd87c-8fe1-4e1b-8d2b-a25bc505e1b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:57.942602 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6bd87c-8fe1-4e1b-8d2b-a25bc505e1b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:57.943112 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6bd87c-8fe1-4e1b-8d2b-a25bc505e1b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:57.943578 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6bd87c-8fe1-4e1b-8d2b-a25bc505e1b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:57.944377 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6bd87c-8fe1-4e1b-8d2b-a25bc505e1b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:57.946212 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6bd87c-8fe1-4e1b-8d2b-a25bc505e1b9 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:57.963669 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1053/2112] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:38:57 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 31 12:38:57.968773 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea498962-3413-4944-8b5e-58cde981cf99 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:57.985073 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea498962-3413-4944-8b5e-58cde981cf99 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:57.997632 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2efcadc-a415-4049-8ef6-118f876120a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:57.997632 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2efcadc-a415-4049-8ef6-118f876120a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:57.997632 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2efcadc-a415-4049-8ef6-118f876120a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:57.997632 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2efcadc-a415-4049-8ef6-118f876120a0 admin admin] PATH_INFO: `/v3/projects/d7feab5558bf4907b13d316a8684a250/users/b041bb66664a4ddf8efeb921db73bc67/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:58.021483 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2efcadc-a415-4049-8ef6-118f876120a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7feab5558bf4907b13d316a8684a250, user_id=b041bb66664a4ddf8efeb921db73bc67, role_id=ca1a5695f8bd499aa0b8cedaeecd2cc6)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:58.025302 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2efcadc-a415-4049-8ef6-118f876120a0 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:58.048831 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1060/2113] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:38:57 2022] PUT /identity/v3/projects/d7feab5558bf4907b13d316a8684a250/users/b041bb66664a4ddf8efeb921db73bc67/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:38:58.063035 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a30b4e-3e7a-4f54-9a2a-ecea9bc7d4ba None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:58.063427 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a30b4e-3e7a-4f54-9a2a-ecea9bc7d4ba None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:58.063840 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a30b4e-3e7a-4f54-9a2a-ecea9bc7d4ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:58.113648 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-10a30b4e-3e7a-4f54-9a2a-ecea9bc7d4ba None None] MFA Rules not processed for user `b041bb66664a4ddf8efeb921db73bc67`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:38:58.114937 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-10a30b4e-3e7a-4f54-9a2a-ecea9bc7d4ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:38:58.208277 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1054/2114] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:38:58 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:38:58.226003 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2efcadc-a415-4049-8ef6-118f876120a0 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:38:58.227346 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-b2efcadc-a415-4049-8ef6-118f876120a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:38:58.303716 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2efcadc-a415-4049-8ef6-118f876120a0 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:38:58.311412 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06ae570-7b48-4fd3-b210-57f635d5cc83 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4feaa01c5ed64b65b10d4dece823702c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:38:58.312342 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06ae570-7b48-4fd3-b210-57f635d5cc83 service glance] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:38:58.312818 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06ae570-7b48-4fd3-b210-57f635d5cc83 service glance] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:38:58.313171 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06ae570-7b48-4fd3-b210-57f635d5cc83 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:38:58.323397 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06ae570-7b48-4fd3-b210-57f635d5cc83 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:38:58.324944 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06ae570-7b48-4fd3-b210-57f635d5cc83 service glance] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:38:58.338882 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1061/2115] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:38:58 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:39:08.015193 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d38b210-87b1-4540-9170-afacc72a2931 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:08.015193 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d38b210-87b1-4540-9170-afacc72a2931 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:08.015773 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d38b210-87b1-4540-9170-afacc72a2931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:08.050297 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-3d38b210-87b1-4540-9170-afacc72a2931 None None] MFA Rules not processed for user `bf4faacf7df248d28f32d7014dadcc07`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:39:08.051631 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-3d38b210-87b1-4540-9170-afacc72a2931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:39:08.091269 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:39:08.129527 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1055/2116] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Oct 31 12:39:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 31 12:39:08.142827 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06ae570-7b48-4fd3-b210-57f635d5cc83 service glance] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:39:08.144595 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-b06ae570-7b48-4fd3-b210-57f635d5cc83 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:39:08.199573 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:39:08.224837 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06ae570-7b48-4fd3-b210-57f635d5cc83 service glance] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:39:08.231123 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c930af70-14dd-4830-91da-35e6446d9068 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2254a5786c249b4aa79244f7198ae65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:39:08.231892 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c930af70-14dd-4830-91da-35e6446d9068 service swift] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:08.232170 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c930af70-14dd-4830-91da-35e6446d9068 service swift] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:08.232445 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c930af70-14dd-4830-91da-35e6446d9068 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:08.242508 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c930af70-14dd-4830-91da-35e6446d9068 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:39:08.244541 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c930af70-14dd-4830-91da-35e6446d9068 service swift] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:39:08.255621 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1062/2117] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Oct 31 12:39:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 31 12:39:16.541432 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d38b210-87b1-4540-9170-afacc72a2931 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:39:16.624925 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d38b210-87b1-4540-9170-afacc72a2931 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:39:16.631289 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b65d8f-8cb3-4cb7-a012-1bc061edcb68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:39:16.632019 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b65d8f-8cb3-4cb7-a012-1bc061edcb68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:16.632386 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b65d8f-8cb3-4cb7-a012-1bc061edcb68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:16.632723 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b65d8f-8cb3-4cb7-a012-1bc061edcb68 admin admin] PATH_INFO: `/v3/users/b041bb66664a4ddf8efeb921db73bc67` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:16.633942 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b65d8f-8cb3-4cb7-a012-1bc061edcb68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b041bb66664a4ddf8efeb921db73bc67)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:39:16.636518 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b65d8f-8cb3-4cb7-a012-1bc061edcb68 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:39:16.697131 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-f5b65d8f-8cb3-4cb7-a012-1bc061edcb68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b041bb66664a4ddf8efeb921db73bc67'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:39:16.709393 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-f5b65d8f-8cb3-4cb7-a012-1bc061edcb68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b041bb66664a4ddf8efeb921db73bc67'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:39:16.710205 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-f5b65d8f-8cb3-4cb7-a012-1bc061edcb68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b041bb66664a4ddf8efeb921db73bc67'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:39:16.740297 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-f5b65d8f-8cb3-4cb7-a012-1bc061edcb68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b041bb66664a4ddf8efeb921db73bc67'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:39:16.748190 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-f5b65d8f-8cb3-4cb7-a012-1bc061edcb68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b041bb66664a4ddf8efeb921db73bc67'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:39:16.754455 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1056/2118] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:39:16 2022] DELETE /identity/v3/users/b041bb66664a4ddf8efeb921db73bc67 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:39:16.759236 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d40dfc-f405-4b04-ab83-46611cd015b9 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:16.759899 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d40dfc-f405-4b04-ab83-46611cd015b9 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:16.759899 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d40dfc-f405-4b04-ab83-46611cd015b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:16.788408 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-76d40dfc-f405-4b04-ab83-46611cd015b9 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:39:16.789734 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-76d40dfc-f405-4b04-ab83-46611cd015b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:39:16.851287 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1063/2119] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:39:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:39:16.860927 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b65d8f-8cb3-4cb7-a012-1bc061edcb68 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:39:16.862442 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-f5b65d8f-8cb3-4cb7-a012-1bc061edcb68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:39:16.929061 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b65d8f-8cb3-4cb7-a012-1bc061edcb68 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:39:16.933956 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed94f7a-7851-484e-9a72-306adc0bb1c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:39:16.934904 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed94f7a-7851-484e-9a72-306adc0bb1c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:16.935395 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed94f7a-7851-484e-9a72-306adc0bb1c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:16.935662 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed94f7a-7851-484e-9a72-306adc0bb1c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:16.945463 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed94f7a-7851-484e-9a72-306adc0bb1c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:39:16.947059 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed94f7a-7851-484e-9a72-306adc0bb1c1 service neutron] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:39:16.958254 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1057/2120] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:39:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:39:17.241129 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d40dfc-f405-4b04-ab83-46611cd015b9 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:39:17.242657 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-76d40dfc-f405-4b04-ab83-46611cd015b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:39:17.259702 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d40dfc-f405-4b04-ab83-46611cd015b9 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:39:17.265058 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b52fe0-cf7b-49ad-9f56-803419b612ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:39:17.265695 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b52fe0-cf7b-49ad-9f56-803419b612ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:17.265967 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b52fe0-cf7b-49ad-9f56-803419b612ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:17.266232 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b52fe0-cf7b-49ad-9f56-803419b612ed admin admin] PATH_INFO: `/v3/projects/d7feab5558bf4907b13d316a8684a250` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:17.268129 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b52fe0-cf7b-49ad-9f56-803419b612ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7feab5558bf4907b13d316a8684a250)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:39:17.270197 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b52fe0-cf7b-49ad-9f56-803419b612ed admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:39:17.322314 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-76b52fe0-cf7b-49ad-9f56-803419b612ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7feab5558bf4907b13d316a8684a250', 'request_id': 'req-76b52fe0-cf7b-49ad-9f56-803419b612ed'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:39:17.328973 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-76b52fe0-cf7b-49ad-9f56-803419b612ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7feab5558bf4907b13d316a8684a250', 'request_id': 'req-76b52fe0-cf7b-49ad-9f56-803419b612ed'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:39:17.335890 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-76b52fe0-cf7b-49ad-9f56-803419b612ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7feab5558bf4907b13d316a8684a250', 'request_id': 'req-76b52fe0-cf7b-49ad-9f56-803419b612ed'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:39:17.347484 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-76b52fe0-cf7b-49ad-9f56-803419b612ed admin admin] The token cache is being invalidate because project d7feab5558bf4907b13d316a8684a250 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:39:17.348039 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-76b52fe0-cf7b-49ad-9f56-803419b612ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:39:17.348811 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-76b52fe0-cf7b-49ad-9f56-803419b612ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:39:17.350697 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1064/2121] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:39:17 2022] DELETE /identity/v3/projects/d7feab5558bf4907b13d316a8684a250 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:39:17.357268 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c87d47d-660b-49b2-8bd0-25d163f211d4 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:17.357595 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c87d47d-660b-49b2-8bd0-25d163f211d4 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:17.357933 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c87d47d-660b-49b2-8bd0-25d163f211d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:17.366992 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:39:17.381168 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-6c87d47d-660b-49b2-8bd0-25d163f211d4 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:39:17.383071 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-6c87d47d-660b-49b2-8bd0-25d163f211d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:39:17.458826 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1058/2122] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:39:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:39:17.466576 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20e8051-4960-425e-a682-4bdccd061f85 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:17.466856 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20e8051-4960-425e-a682-4bdccd061f85 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:17.467125 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20e8051-4960-425e-a682-4bdccd061f85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:17.492061 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-d20e8051-4960-425e-a682-4bdccd061f85 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:39:17.493273 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-d20e8051-4960-425e-a682-4bdccd061f85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:39:17.502278 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1065/2123] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:39:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:39:17.506507 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c87d47d-660b-49b2-8bd0-25d163f211d4 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:39:17.516511 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c87d47d-660b-49b2-8bd0-25d163f211d4 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:39:17.523140 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baa07e9-4c0b-44ad-8368-c019812c55f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:39:17.524056 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baa07e9-4c0b-44ad-8368-c019812c55f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:17.524573 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baa07e9-4c0b-44ad-8368-c019812c55f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:17.524850 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baa07e9-4c0b-44ad-8368-c019812c55f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:17.525498 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baa07e9-4c0b-44ad-8368-c019812c55f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:39:17.525946 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baa07e9-4c0b-44ad-8368-c019812c55f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:39:17.529546 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baa07e9-4c0b-44ad-8368-c019812c55f6 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:39:17.543199 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1059/2124] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Oct 31 12:39:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 31 12:39:17.546807 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20e8051-4960-425e-a682-4bdccd061f85 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:39:17.565571 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20e8051-4960-425e-a682-4bdccd061f85 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:39:17.573546 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97aa3a3-4179-47d1-9cd4-533ff364a3c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:39:17.574377 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97aa3a3-4179-47d1-9cd4-533ff364a3c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:17.574751 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97aa3a3-4179-47d1-9cd4-533ff364a3c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:17.575136 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97aa3a3-4179-47d1-9cd4-533ff364a3c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:17.575869 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97aa3a3-4179-47d1-9cd4-533ff364a3c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:39:17.578090 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97aa3a3-4179-47d1-9cd4-533ff364a3c8 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:39:17.609766 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1066/2125] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:39:17 2022] POST /identity/v3/projects => generated 375 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:39:17.614035 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baa07e9-4c0b-44ad-8368-c019812c55f6 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:39:17.624670 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baa07e9-4c0b-44ad-8368-c019812c55f6 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:39:17.680241 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c8a826-feac-4968-b3c9-75d9d3ed63d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:39:17.680957 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c8a826-feac-4968-b3c9-75d9d3ed63d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:17.681233 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c8a826-feac-4968-b3c9-75d9d3ed63d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:17.681527 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c8a826-feac-4968-b3c9-75d9d3ed63d7 admin admin] PATH_INFO: `/v3/users` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:17.682159 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c8a826-feac-4968-b3c9-75d9d3ed63d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:39:17.684602 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c8a826-feac-4968-b3c9-75d9d3ed63d7 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:39:17.707006 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1060/2126] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:39:17 2022] POST /identity/v3/users => generated 288 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:39:17.710464 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97aa3a3-4179-47d1-9cd4-533ff364a3c8 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:39:17.720742 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97aa3a3-4179-47d1-9cd4-533ff364a3c8 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:39:17.725306 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8854440a-298b-4168-a85b-37bca23365e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:39:17.725900 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8854440a-298b-4168-a85b-37bca23365e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:17.726175 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8854440a-298b-4168-a85b-37bca23365e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:17.726443 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8854440a-298b-4168-a85b-37bca23365e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:17.727071 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8854440a-298b-4168-a85b-37bca23365e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:39:17.728673 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8854440a-298b-4168-a85b-37bca23365e6 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:39:17.738837 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1067/2127] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:39:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:39:17.742791 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c8a826-feac-4968-b3c9-75d9d3ed63d7 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:39:17.753349 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c8a826-feac-4968-b3c9-75d9d3ed63d7 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:39:17.757906 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080e9715-67b4-431e-aa51-6520020a8f65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:39:17.758490 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080e9715-67b4-431e-aa51-6520020a8f65 admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:17.758769 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080e9715-67b4-431e-aa51-6520020a8f65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:17.759039 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080e9715-67b4-431e-aa51-6520020a8f65 admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:17.759717 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080e9715-67b4-431e-aa51-6520020a8f65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:39:17.761981 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080e9715-67b4-431e-aa51-6520020a8f65 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:39:17.772725 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1061/2128] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:39:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:39:17.776866 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8854440a-298b-4168-a85b-37bca23365e6 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:39:17.788847 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8854440a-298b-4168-a85b-37bca23365e6 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:39:17.795108 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa8a46c-c7ee-4fde-8d3d-8e5c11286d68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:39:17.796122 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa8a46c-c7ee-4fde-8d3d-8e5c11286d68 admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:17.796369 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa8a46c-c7ee-4fde-8d3d-8e5c11286d68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:17.796725 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa8a46c-c7ee-4fde-8d3d-8e5c11286d68 admin admin] PATH_INFO: `/v3/projects/741f915aad03472e8373166c4d7fd380/users/66eb8a35b8b64ad0adaa54971706420e/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:17.812178 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa8a46c-c7ee-4fde-8d3d-8e5c11286d68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=741f915aad03472e8373166c4d7fd380, user_id=66eb8a35b8b64ad0adaa54971706420e, role_id=ca1a5695f8bd499aa0b8cedaeecd2cc6)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:39:17.814940 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa8a46c-c7ee-4fde-8d3d-8e5c11286d68 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:39:17.818181 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:39:17.830174 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1068/2129] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:39:17 2022] PUT /identity/v3/projects/741f915aad03472e8373166c4d7fd380/users/66eb8a35b8b64ad0adaa54971706420e/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 31 12:39:17.843008 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d166eb6b-2b5e-499c-9702-09d69a5c4d5b None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:17.843461 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d166eb6b-2b5e-499c-9702-09d69a5c4d5b None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:17.843744 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d166eb6b-2b5e-499c-9702-09d69a5c4d5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:17.872391 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-d166eb6b-2b5e-499c-9702-09d69a5c4d5b None None] MFA Rules not processed for user `66eb8a35b8b64ad0adaa54971706420e`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:39:17.873556 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-d166eb6b-2b5e-499c-9702-09d69a5c4d5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:39:17.935069 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:39:17.939612 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1062/2130] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:39:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 31 12:39:17.951344 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa8a46c-c7ee-4fde-8d3d-8e5c11286d68 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:39:17.953091 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-6aa8a46c-c7ee-4fde-8d3d-8e5c11286d68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:39:18.009400 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa8a46c-c7ee-4fde-8d3d-8e5c11286d68 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:39:18.013931 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145913c6-0d48-4311-9f35-4cf2b7fd6878 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d6d7ea077d49e8a66c6c6762499702', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:39:18.014531 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145913c6-0d48-4311-9f35-4cf2b7fd6878 service cinder] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:18.014807 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145913c6-0d48-4311-9f35-4cf2b7fd6878 service cinder] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:18.015078 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145913c6-0d48-4311-9f35-4cf2b7fd6878 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:18.024003 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145913c6-0d48-4311-9f35-4cf2b7fd6878 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:39:18.025740 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145913c6-0d48-4311-9f35-4cf2b7fd6878 service cinder] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:39:18.036225 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1069/2131] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:39:17 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:39:19.430648 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad7d4d8-0935-4715-a610-2859de0bfe7b None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:19.431193 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad7d4d8-0935-4715-a610-2859de0bfe7b None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:19.431289 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad7d4d8-0935-4715-a610-2859de0bfe7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:19.453096 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-6ad7d4d8-0935-4715-a610-2859de0bfe7b None None] MFA Rules not processed for user `66eb8a35b8b64ad0adaa54971706420e`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:39:19.454286 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-6ad7d4d8-0935-4715-a610-2859de0bfe7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:39:19.461439 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1063/2132] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:39:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:39:24.129031 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145913c6-0d48-4311-9f35-4cf2b7fd6878 service cinder] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:39:24.130652 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-145913c6-0d48-4311-9f35-4cf2b7fd6878 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:39:24.196662 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145913c6-0d48-4311-9f35-4cf2b7fd6878 service cinder] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:39:24.200873 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2323b4-cb24-49e9-991b-2200df404c43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:39:24.201478 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2323b4-cb24-49e9-991b-2200df404c43 service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:24.201750 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2323b4-cb24-49e9-991b-2200df404c43 service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:24.202017 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2323b4-cb24-49e9-991b-2200df404c43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:24.203775 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-ba2323b4-cb24-49e9-991b-2200df404c43 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:39:24.265119 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2323b4-cb24-49e9-991b-2200df404c43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:39:24.266647 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2323b4-cb24-49e9-991b-2200df404c43 service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:39:24.276592 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1070/2133] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:39:24 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:39:28.919512 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f288bb08-b398-483f-8f9e-902b2aa197b5 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:28.920018 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f288bb08-b398-483f-8f9e-902b2aa197b5 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:28.920179 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f288bb08-b398-483f-8f9e-902b2aa197b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:28.948651 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-f288bb08-b398-483f-8f9e-902b2aa197b5 None None] MFA Rules not processed for user `bf4faacf7df248d28f32d7014dadcc07`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:39:28.950757 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-f288bb08-b398-483f-8f9e-902b2aa197b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:39:29.013104 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1064/2134] 10.222.0.15 () {58 vars in 967 bytes} [Mon Oct 31 12:39:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:39:29.039558 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2323b4-cb24-49e9-991b-2200df404c43 service neutron] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:39:29.041043 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-ba2323b4-cb24-49e9-991b-2200df404c43 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:39:29.096218 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:39:29.117794 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2323b4-cb24-49e9-991b-2200df404c43 service neutron] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:39:29.123627 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4008ffcb-6694-4da0-91ab-95fba015f93c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2254a5786c249b4aa79244f7198ae65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:39:29.124317 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4008ffcb-6694-4da0-91ab-95fba015f93c service swift] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:29.124665 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4008ffcb-6694-4da0-91ab-95fba015f93c service swift] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:29.125006 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4008ffcb-6694-4da0-91ab-95fba015f93c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:29.133608 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4008ffcb-6694-4da0-91ab-95fba015f93c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:39:29.135208 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4008ffcb-6694-4da0-91ab-95fba015f93c service swift] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:39:29.146478 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1071/2135] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Oct 31 12:39:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 31 12:39:34.687126 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f288bb08-b398-483f-8f9e-902b2aa197b5 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:39:34.689481 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-f288bb08-b398-483f-8f9e-902b2aa197b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:39:34.701382 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f288bb08-b398-483f-8f9e-902b2aa197b5 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:39:34.705410 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747f59b9-8f6c-499a-aecc-12576a4409f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:39:34.706011 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747f59b9-8f6c-499a-aecc-12576a4409f5 service neutron] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:34.706463 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747f59b9-8f6c-499a-aecc-12576a4409f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:34.706731 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747f59b9-8f6c-499a-aecc-12576a4409f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:34.708756 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-747f59b9-8f6c-499a-aecc-12576a4409f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:39:34.780166 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747f59b9-8f6c-499a-aecc-12576a4409f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:39:34.781776 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747f59b9-8f6c-499a-aecc-12576a4409f5 service neutron] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:39:34.794090 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1065/2136] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:39:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:39:38.497586 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4008ffcb-6694-4da0-91ab-95fba015f93c service swift] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:39:38.499460 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-4008ffcb-6694-4da0-91ab-95fba015f93c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:39:38.611783 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4008ffcb-6694-4da0-91ab-95fba015f93c service swift] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:39:38.633341 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e55d86-f357-4519-aa59-9b167f62427a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e7777495d7a406c8c50b1ad52e47263', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:39:38.634285 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e55d86-f357-4519-aa59-9b167f62427a service placement] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:38.635847 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e55d86-f357-4519-aa59-9b167f62427a service placement] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:38.636344 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e55d86-f357-4519-aa59-9b167f62427a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:38.638828 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-92e55d86-f357-4519-aa59-9b167f62427a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:39:38.659778 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e55d86-f357-4519-aa59-9b167f62427a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:39:38.659778 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e55d86-f357-4519-aa59-9b167f62427a service placement] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:39:38.670962 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1072/2137] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:39:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:39:40.757299 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f4a5f4-4397-4b44-8674-a7f69f42f929 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:40.757871 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f4a5f4-4397-4b44-8674-a7f69f42f929 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:40.757871 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f4a5f4-4397-4b44-8674-a7f69f42f929 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:40.799604 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-20f4a5f4-4397-4b44-8674-a7f69f42f929 None None] MFA Rules not processed for user `bf4faacf7df248d28f32d7014dadcc07`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:39:40.801232 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-20f4a5f4-4397-4b44-8674-a7f69f42f929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:39:40.821150 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1066/2138] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Oct 31 12:39:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 31 12:39:40.842476 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e55d86-f357-4519-aa59-9b167f62427a service placement] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:39:40.860223 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e55d86-f357-4519-aa59-9b167f62427a service placement] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:39:40.873185 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0d40b2-f42c-4fad-a7f3-881213765936 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2254a5786c249b4aa79244f7198ae65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:39:40.875154 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0d40b2-f42c-4fad-a7f3-881213765936 service swift] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:40.875760 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0d40b2-f42c-4fad-a7f3-881213765936 service swift] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:40.876293 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0d40b2-f42c-4fad-a7f3-881213765936 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:40.888405 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0d40b2-f42c-4fad-a7f3-881213765936 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:39:40.890297 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0d40b2-f42c-4fad-a7f3-881213765936 service swift] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:39:40.901465 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1073/2139] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Oct 31 12:39:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 31 12:39:41.866501 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b971e1-b69d-4663-8843-75c03335c191 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:41.867448 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b971e1-b69d-4663-8843-75c03335c191 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:41.867871 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b971e1-b69d-4663-8843-75c03335c191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:41.896549 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-f5b971e1-b69d-4663-8843-75c03335c191 None None] MFA Rules not processed for user `bf4faacf7df248d28f32d7014dadcc07`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:39:41.899107 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-f5b971e1-b69d-4663-8843-75c03335c191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:39:41.910125 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1067/2140] 10.222.0.15 () {58 vars in 967 bytes} [Mon Oct 31 12:39:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:39:41.929615 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0d40b2-f42c-4fad-a7f3-881213765936 service swift] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:39:41.942166 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0d40b2-f42c-4fad-a7f3-881213765936 service swift] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:39:41.948067 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa568485-f10f-478c-91e1-04e41cdd655c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2254a5786c249b4aa79244f7198ae65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:39:41.949416 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa568485-f10f-478c-91e1-04e41cdd655c service swift] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:41.949881 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa568485-f10f-478c-91e1-04e41cdd655c service swift] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:41.950320 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa568485-f10f-478c-91e1-04e41cdd655c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:41.959662 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa568485-f10f-478c-91e1-04e41cdd655c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:39:41.961398 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa568485-f10f-478c-91e1-04e41cdd655c service swift] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:39:41.972443 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1074/2141] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Oct 31 12:39:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 31 12:39:42.276877 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0115a751-420d-4edf-aaab-94e9387bd1b3 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:42.277274 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0115a751-420d-4edf-aaab-94e9387bd1b3 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:42.277626 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0115a751-420d-4edf-aaab-94e9387bd1b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:42.304147 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-0115a751-420d-4edf-aaab-94e9387bd1b3 None None] MFA Rules not processed for user `bf4faacf7df248d28f32d7014dadcc07`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:39:42.305359 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-0115a751-420d-4edf-aaab-94e9387bd1b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:39:42.313796 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1068/2142] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Oct 31 12:39:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:39:42.327418 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa568485-f10f-478c-91e1-04e41cdd655c service swift] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:39:42.338046 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa568485-f10f-478c-91e1-04e41cdd655c service swift] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:39:42.343975 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1646c47-ee1f-41b9-b9dc-dda4fb5139da service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2254a5786c249b4aa79244f7198ae65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:39:42.344582 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1646c47-ee1f-41b9-b9dc-dda4fb5139da service swift] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:42.344860 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1646c47-ee1f-41b9-b9dc-dda4fb5139da service swift] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:42.345132 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1646c47-ee1f-41b9-b9dc-dda4fb5139da service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:42.353568 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1646c47-ee1f-41b9-b9dc-dda4fb5139da service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:39:42.355902 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1646c47-ee1f-41b9-b9dc-dda4fb5139da service swift] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:39:42.364017 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1075/2143] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Oct 31 12:39:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 31 12:39:42.731703 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceb71f2-9103-4ad7-b0c2-98294c439936 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:42.732217 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceb71f2-9103-4ad7-b0c2-98294c439936 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:42.732638 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceb71f2-9103-4ad7-b0c2-98294c439936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:42.765451 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-dceb71f2-9103-4ad7-b0c2-98294c439936 None None] MFA Rules not processed for user `bf4faacf7df248d28f32d7014dadcc07`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:39:42.766849 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-dceb71f2-9103-4ad7-b0c2-98294c439936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:39:42.782295 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1069/2144] 10.222.0.15 () {58 vars in 967 bytes} [Mon Oct 31 12:39:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:39:42.798690 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1646c47-ee1f-41b9-b9dc-dda4fb5139da service swift] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:39:42.819515 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1646c47-ee1f-41b9-b9dc-dda4fb5139da service swift] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:39:42.842700 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc4c29a-b177-4945-860b-45d2f882a587 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2254a5786c249b4aa79244f7198ae65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:39:42.842700 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc4c29a-b177-4945-860b-45d2f882a587 service swift] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:42.842700 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc4c29a-b177-4945-860b-45d2f882a587 service swift] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:42.842700 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc4c29a-b177-4945-860b-45d2f882a587 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:42.852107 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc4c29a-b177-4945-860b-45d2f882a587 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:39:42.854209 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc4c29a-b177-4945-860b-45d2f882a587 service swift] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:39:42.874532 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1076/2145] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Oct 31 12:39:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 31 12:39:44.060115 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceb71f2-9103-4ad7-b0c2-98294c439936 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:39:44.063973 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-dceb71f2-9103-4ad7-b0c2-98294c439936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:39:44.088565 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceb71f2-9103-4ad7-b0c2-98294c439936 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:39:44.094082 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e6c187-ca14-42af-bf7d-7af4ce62ca1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:39:44.094743 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e6c187-ca14-42af-bf7d-7af4ce62ca1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:44.095494 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e6c187-ca14-42af-bf7d-7af4ce62ca1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:44.095953 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e6c187-ca14-42af-bf7d-7af4ce62ca1b admin admin] PATH_INFO: `/v3/users/825059319a204b6da5bc7004b3a09361` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:44.098352 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e6c187-ca14-42af-bf7d-7af4ce62ca1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=825059319a204b6da5bc7004b3a09361)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:39:44.101429 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e6c187-ca14-42af-bf7d-7af4ce62ca1b admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:39:44.192921 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-b5e6c187-ca14-42af-bf7d-7af4ce62ca1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '825059319a204b6da5bc7004b3a09361'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:39:44.209870 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-b5e6c187-ca14-42af-bf7d-7af4ce62ca1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '825059319a204b6da5bc7004b3a09361'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:39:44.210734 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-b5e6c187-ca14-42af-bf7d-7af4ce62ca1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '825059319a204b6da5bc7004b3a09361'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:39:44.222603 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:39:44.245883 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-b5e6c187-ca14-42af-bf7d-7af4ce62ca1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '825059319a204b6da5bc7004b3a09361'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:39:44.255752 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-b5e6c187-ca14-42af-bf7d-7af4ce62ca1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '825059319a204b6da5bc7004b3a09361'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:39:44.265738 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1070/2146] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:39:44 2022] DELETE /identity/v3/users/825059319a204b6da5bc7004b3a09361 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:39:44.488777 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc4c29a-b177-4945-860b-45d2f882a587 service swift] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:39:44.503734 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc4c29a-b177-4945-860b-45d2f882a587 service swift] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:39:44.567145 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36217a0-2dbb-4950-93c4-0300f0206bac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:39:44.567911 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36217a0-2dbb-4950-93c4-0300f0206bac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:44.568185 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36217a0-2dbb-4950-93c4-0300f0206bac admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:44.568452 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36217a0-2dbb-4950-93c4-0300f0206bac admin admin] PATH_INFO: `/v3/projects/b14b98f006574deabc030c2e93c680c4` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:44.569647 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36217a0-2dbb-4950-93c4-0300f0206bac admin admin] RBAC: Authorizing `identity:delete_project(project_id=b14b98f006574deabc030c2e93c680c4)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:39:44.571722 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36217a0-2dbb-4950-93c4-0300f0206bac admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:39:44.596783 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:39:44.620550 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-b36217a0-2dbb-4950-93c4-0300f0206bac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b14b98f006574deabc030c2e93c680c4', 'request_id': 'req-b36217a0-2dbb-4950-93c4-0300f0206bac'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:39:44.626823 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-b36217a0-2dbb-4950-93c4-0300f0206bac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b14b98f006574deabc030c2e93c680c4', 'request_id': 'req-b36217a0-2dbb-4950-93c4-0300f0206bac'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:39:44.633648 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-b36217a0-2dbb-4950-93c4-0300f0206bac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b14b98f006574deabc030c2e93c680c4', 'request_id': 'req-b36217a0-2dbb-4950-93c4-0300f0206bac'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:39:44.645387 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-b36217a0-2dbb-4950-93c4-0300f0206bac admin admin] The token cache is being invalidate because project b14b98f006574deabc030c2e93c680c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:39:44.645686 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-b36217a0-2dbb-4950-93c4-0300f0206bac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:39:44.646445 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-b36217a0-2dbb-4950-93c4-0300f0206bac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:39:44.648203 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1077/2147] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:39:44 2022] DELETE /identity/v3/projects/b14b98f006574deabc030c2e93c680c4 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:39:44.652925 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e6c187-ca14-42af-bf7d-7af4ce62ca1b admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:39:44.654639 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-b5e6c187-ca14-42af-bf7d-7af4ce62ca1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:39:44.670161 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e6c187-ca14-42af-bf7d-7af4ce62ca1b admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:39:44.732907 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9cc9ed-3f84-4bff-b362-99b45ecf775d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:39:44.733467 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9cc9ed-3f84-4bff-b362-99b45ecf775d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:44.733743 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9cc9ed-3f84-4bff-b362-99b45ecf775d admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:44.734012 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9cc9ed-3f84-4bff-b362-99b45ecf775d admin admin] PATH_INFO: `/v3/users/aa1cbcd62cb04c6a9258474764721df0` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:44.735443 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9cc9ed-3f84-4bff-b362-99b45ecf775d admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa1cbcd62cb04c6a9258474764721df0)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:39:44.738174 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9cc9ed-3f84-4bff-b362-99b45ecf775d admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:39:44.784693 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-7f9cc9ed-3f84-4bff-b362-99b45ecf775d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa1cbcd62cb04c6a9258474764721df0'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:39:44.793034 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-7f9cc9ed-3f84-4bff-b362-99b45ecf775d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa1cbcd62cb04c6a9258474764721df0'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:39:44.793906 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-7f9cc9ed-3f84-4bff-b362-99b45ecf775d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa1cbcd62cb04c6a9258474764721df0'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:39:44.805878 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-7f9cc9ed-3f84-4bff-b362-99b45ecf775d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa1cbcd62cb04c6a9258474764721df0'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:39:44.812929 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-7f9cc9ed-3f84-4bff-b362-99b45ecf775d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa1cbcd62cb04c6a9258474764721df0'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:39:44.818724 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1071/2148] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:39:44 2022] DELETE /identity/v3/users/aa1cbcd62cb04c6a9258474764721df0 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:39:44.991804 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36217a0-2dbb-4950-93c4-0300f0206bac admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:39:44.993311 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-b36217a0-2dbb-4950-93c4-0300f0206bac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:39:45.007036 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36217a0-2dbb-4950-93c4-0300f0206bac admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:39:45.065676 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb44b3f8-f4e0-445a-a315-443566d7e772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:39:45.066305 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb44b3f8-f4e0-445a-a315-443566d7e772 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:45.066580 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb44b3f8-f4e0-445a-a315-443566d7e772 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:45.066842 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb44b3f8-f4e0-445a-a315-443566d7e772 admin admin] PATH_INFO: `/v3/projects/656dea667748432aadfffc81fb3a8cb3` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:45.067980 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb44b3f8-f4e0-445a-a315-443566d7e772 admin admin] RBAC: Authorizing `identity:delete_project(project_id=656dea667748432aadfffc81fb3a8cb3)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:39:45.069843 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb44b3f8-f4e0-445a-a315-443566d7e772 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:39:45.116741 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-eb44b3f8-f4e0-445a-a315-443566d7e772 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '656dea667748432aadfffc81fb3a8cb3', 'request_id': 'req-eb44b3f8-f4e0-445a-a315-443566d7e772'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:39:45.123414 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-eb44b3f8-f4e0-445a-a315-443566d7e772 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '656dea667748432aadfffc81fb3a8cb3', 'request_id': 'req-eb44b3f8-f4e0-445a-a315-443566d7e772'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:39:45.130405 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-eb44b3f8-f4e0-445a-a315-443566d7e772 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '656dea667748432aadfffc81fb3a8cb3', 'request_id': 'req-eb44b3f8-f4e0-445a-a315-443566d7e772'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:39:45.142356 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-eb44b3f8-f4e0-445a-a315-443566d7e772 admin admin] The token cache is being invalidate because project 656dea667748432aadfffc81fb3a8cb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:39:45.142837 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-eb44b3f8-f4e0-445a-a315-443566d7e772 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:39:45.143746 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-eb44b3f8-f4e0-445a-a315-443566d7e772 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:39:45.145217 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1078/2149] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:39:44 2022] DELETE /identity/v3/projects/656dea667748432aadfffc81fb3a8cb3 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:39:45.475039 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9cc9ed-3f84-4bff-b362-99b45ecf775d admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:39:45.477898 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-7f9cc9ed-3f84-4bff-b362-99b45ecf775d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:39:45.498079 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9cc9ed-3f84-4bff-b362-99b45ecf775d admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:39:45.525138 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb44b3f8-f4e0-445a-a315-443566d7e772 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:39:45.527065 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-eb44b3f8-f4e0-445a-a315-443566d7e772 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:39:45.571609 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:39:45.571829 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:39:45.580808 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f3f7b1-9a77-4228-8dde-69330753982c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:39:45.581818 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f3f7b1-9a77-4228-8dde-69330753982c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:45.582086 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f3f7b1-9a77-4228-8dde-69330753982c admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:45.582470 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f3f7b1-9a77-4228-8dde-69330753982c admin admin] PATH_INFO: `/v3/users/66eb8a35b8b64ad0adaa54971706420e` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:45.583885 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f3f7b1-9a77-4228-8dde-69330753982c admin admin] RBAC: Authorizing `identity:delete_user(user_id=66eb8a35b8b64ad0adaa54971706420e)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:39:45.586394 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f3f7b1-9a77-4228-8dde-69330753982c admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:39:45.590153 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb44b3f8-f4e0-445a-a315-443566d7e772 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:39:45.594055 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0e442f-02b0-46c1-b21c-1968a649f458 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:39:45.594657 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0e442f-02b0-46c1-b21c-1968a649f458 service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:45.594934 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0e442f-02b0-46c1-b21c-1968a649f458 service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:45.595234 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0e442f-02b0-46c1-b21c-1968a649f458 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:45.596973 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-df0e442f-02b0-46c1-b21c-1968a649f458 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:39:45.610206 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0e442f-02b0-46c1-b21c-1968a649f458 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:39:45.612053 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0e442f-02b0-46c1-b21c-1968a649f458 service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:39:45.623476 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1079/2150] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:39:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 31 12:39:45.636428 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-92f3f7b1-9a77-4228-8dde-69330753982c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66eb8a35b8b64ad0adaa54971706420e'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:39:45.644880 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-92f3f7b1-9a77-4228-8dde-69330753982c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66eb8a35b8b64ad0adaa54971706420e'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:39:45.645681 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-92f3f7b1-9a77-4228-8dde-69330753982c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66eb8a35b8b64ad0adaa54971706420e'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:39:45.659386 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-92f3f7b1-9a77-4228-8dde-69330753982c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66eb8a35b8b64ad0adaa54971706420e'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:39:45.667824 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-92f3f7b1-9a77-4228-8dde-69330753982c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66eb8a35b8b64ad0adaa54971706420e'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:39:45.674536 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1072/2151] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:39:45 2022] DELETE /identity/v3/users/66eb8a35b8b64ad0adaa54971706420e => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:39:45.679284 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f11290-42e5-44ca-ab0a-89c88f808783 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:45.679688 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f11290-42e5-44ca-ab0a-89c88f808783 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:45.679971 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f11290-42e5-44ca-ab0a-89c88f808783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:45.702121 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-56f11290-42e5-44ca-ab0a-89c88f808783 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:39:45.703358 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-56f11290-42e5-44ca-ab0a-89c88f808783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:39:45.763462 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1080/2152] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:39:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 31 12:39:45.774435 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f3f7b1-9a77-4228-8dde-69330753982c admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:39:45.775977 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-92f3f7b1-9a77-4228-8dde-69330753982c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:39:45.838798 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f3f7b1-9a77-4228-8dde-69330753982c admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:39:45.842533 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f433d0-1bac-4d4d-9b21-c3e6f9ef5180 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:39:45.843132 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f433d0-1bac-4d4d-9b21-c3e6f9ef5180 service neutron] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:45.843442 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f433d0-1bac-4d4d-9b21-c3e6f9ef5180 service neutron] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:45.843714 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f433d0-1bac-4d4d-9b21-c3e6f9ef5180 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:45.853190 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f433d0-1bac-4d4d-9b21-c3e6f9ef5180 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:39:45.855708 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f433d0-1bac-4d4d-9b21-c3e6f9ef5180 service neutron] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:39:45.865712 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1073/2153] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:39:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:39:46.077283 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f11290-42e5-44ca-ab0a-89c88f808783 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:39:46.078777 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-56f11290-42e5-44ca-ab0a-89c88f808783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:39:46.094024 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f11290-42e5-44ca-ab0a-89c88f808783 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:39:46.099140 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fa63d4-0f05-49ac-88cd-f959ec5e6672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:39:46.100105 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fa63d4-0f05-49ac-88cd-f959ec5e6672 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:46.100625 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fa63d4-0f05-49ac-88cd-f959ec5e6672 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:46.101095 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fa63d4-0f05-49ac-88cd-f959ec5e6672 admin admin] PATH_INFO: `/v3/projects/741f915aad03472e8373166c4d7fd380` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:46.102437 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fa63d4-0f05-49ac-88cd-f959ec5e6672 admin admin] RBAC: Authorizing `identity:delete_project(project_id=741f915aad03472e8373166c4d7fd380)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:39:46.105543 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fa63d4-0f05-49ac-88cd-f959ec5e6672 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:39:46.162933 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-e6fa63d4-0f05-49ac-88cd-f959ec5e6672 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '741f915aad03472e8373166c4d7fd380', 'request_id': 'req-e6fa63d4-0f05-49ac-88cd-f959ec5e6672'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:39:46.170823 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-e6fa63d4-0f05-49ac-88cd-f959ec5e6672 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '741f915aad03472e8373166c4d7fd380', 'request_id': 'req-e6fa63d4-0f05-49ac-88cd-f959ec5e6672'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:39:46.179639 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-e6fa63d4-0f05-49ac-88cd-f959ec5e6672 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '741f915aad03472e8373166c4d7fd380', 'request_id': 'req-e6fa63d4-0f05-49ac-88cd-f959ec5e6672'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:39:46.191588 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-e6fa63d4-0f05-49ac-88cd-f959ec5e6672 admin admin] The token cache is being invalidate because project 741f915aad03472e8373166c4d7fd380 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:39:46.192040 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-e6fa63d4-0f05-49ac-88cd-f959ec5e6672 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:39:46.192765 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-e6fa63d4-0f05-49ac-88cd-f959ec5e6672 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:39:46.194455 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1081/2154] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:39:46 2022] DELETE /identity/v3/projects/741f915aad03472e8373166c4d7fd380 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:39:46.619878 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f433d0-1bac-4d4d-9b21-c3e6f9ef5180 service neutron] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:39:46.629118 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f433d0-1bac-4d4d-9b21-c3e6f9ef5180 service neutron] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:39:46.686254 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af185278-f13a-4310-90a6-de8d563a135d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:39:46.686935 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af185278-f13a-4310-90a6-de8d563a135d service neutron] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:46.687244 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af185278-f13a-4310-90a6-de8d563a135d service neutron] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:46.687518 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af185278-f13a-4310-90a6-de8d563a135d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:46.689386 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-af185278-f13a-4310-90a6-de8d563a135d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:39:46.731766 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:39:46.771201 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af185278-f13a-4310-90a6-de8d563a135d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:39:46.772665 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af185278-f13a-4310-90a6-de8d563a135d service neutron] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:39:46.783828 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1074/2155] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:39:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:39:49.262095 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fa63d4-0f05-49ac-88cd-f959ec5e6672 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:39:49.264159 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-e6fa63d4-0f05-49ac-88cd-f959ec5e6672 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:39:49.287259 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fa63d4-0f05-49ac-88cd-f959ec5e6672 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:39:49.292514 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f2ea27-d9a9-4f1a-a71c-0507f4bf0e8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:39:49.293581 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f2ea27-d9a9-4f1a-a71c-0507f4bf0e8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:49.294122 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f2ea27-d9a9-4f1a-a71c-0507f4bf0e8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:49.294560 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f2ea27-d9a9-4f1a-a71c-0507f4bf0e8f admin admin] PATH_INFO: `/v3/users/b2c03f0365ee4a26b8c076ae365325b2` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:49.296240 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f2ea27-d9a9-4f1a-a71c-0507f4bf0e8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2c03f0365ee4a26b8c076ae365325b2)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:39:49.298346 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f2ea27-d9a9-4f1a-a71c-0507f4bf0e8f admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:39:49.369614 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-d6f2ea27-d9a9-4f1a-a71c-0507f4bf0e8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2c03f0365ee4a26b8c076ae365325b2'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:39:49.386134 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-d6f2ea27-d9a9-4f1a-a71c-0507f4bf0e8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2c03f0365ee4a26b8c076ae365325b2'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:39:49.386820 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-d6f2ea27-d9a9-4f1a-a71c-0507f4bf0e8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2c03f0365ee4a26b8c076ae365325b2'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:39:49.387511 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-d6f2ea27-d9a9-4f1a-a71c-0507f4bf0e8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2c03f0365ee4a26b8c076ae365325b2'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:39:49.395919 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:39:49.438040 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-d6f2ea27-d9a9-4f1a-a71c-0507f4bf0e8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2c03f0365ee4a26b8c076ae365325b2'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:39:49.456856 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1082/2156] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:39:49 2022] DELETE /identity/v3/users/b2c03f0365ee4a26b8c076ae365325b2 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:39:49.647817 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af185278-f13a-4310-90a6-de8d563a135d service neutron] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:39:49.660343 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af185278-f13a-4310-90a6-de8d563a135d service neutron] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:39:49.724600 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dbb39f-bca2-4319-be6c-2830d993c225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:39:49.725259 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dbb39f-bca2-4319-be6c-2830d993c225 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:49.725553 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dbb39f-bca2-4319-be6c-2830d993c225 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:49.727547 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dbb39f-bca2-4319-be6c-2830d993c225 admin admin] PATH_INFO: `/v3/projects/bfbc3a46f12942299b99cdd20c989e1f` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:49.731449 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dbb39f-bca2-4319-be6c-2830d993c225 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfbc3a46f12942299b99cdd20c989e1f)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:39:49.733784 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dbb39f-bca2-4319-be6c-2830d993c225 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:39:49.807688 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-90dbb39f-bca2-4319-be6c-2830d993c225 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfbc3a46f12942299b99cdd20c989e1f', 'request_id': 'req-90dbb39f-bca2-4319-be6c-2830d993c225'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:39:49.815681 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-90dbb39f-bca2-4319-be6c-2830d993c225 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfbc3a46f12942299b99cdd20c989e1f', 'request_id': 'req-90dbb39f-bca2-4319-be6c-2830d993c225'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:39:49.826088 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-90dbb39f-bca2-4319-be6c-2830d993c225 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfbc3a46f12942299b99cdd20c989e1f', 'request_id': 'req-90dbb39f-bca2-4319-be6c-2830d993c225'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:39:49.843574 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-90dbb39f-bca2-4319-be6c-2830d993c225 admin admin] The token cache is being invalidate because project bfbc3a46f12942299b99cdd20c989e1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:39:49.843924 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-90dbb39f-bca2-4319-be6c-2830d993c225 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:39:49.844615 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-90dbb39f-bca2-4319-be6c-2830d993c225 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:39:49.846324 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1075/2157] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:39:49 2022] DELETE /identity/v3/projects/bfbc3a46f12942299b99cdd20c989e1f => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:39:49.851218 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969d73ef-274c-446e-baf5-b29c0dc92cd5 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:49.852718 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969d73ef-274c-446e-baf5-b29c0dc92cd5 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:49.852981 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969d73ef-274c-446e-baf5-b29c0dc92cd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:49.883530 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-969d73ef-274c-446e-baf5-b29c0dc92cd5 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:39:49.887579 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-969d73ef-274c-446e-baf5-b29c0dc92cd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:39:49.972974 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1083/2158] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:39:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:39:49.980803 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbcf766-7dec-4321-b417-0fd729c85849 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:49.981142 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbcf766-7dec-4321-b417-0fd729c85849 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:49.981301 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbcf766-7dec-4321-b417-0fd729c85849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:50.009656 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-acbcf766-7dec-4321-b417-0fd729c85849 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:39:50.011019 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-acbcf766-7dec-4321-b417-0fd729c85849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:39:50.027376 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1076/2159] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:39:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:39:50.029068 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969d73ef-274c-446e-baf5-b29c0dc92cd5 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:39:50.047858 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969d73ef-274c-446e-baf5-b29c0dc92cd5 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:39:50.054368 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555a85c1-4c52-4f43-a4a5-1ba3b8307f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:39:50.054890 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555a85c1-4c52-4f43-a4a5-1ba3b8307f71 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:50.055188 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555a85c1-4c52-4f43-a4a5-1ba3b8307f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:50.055466 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555a85c1-4c52-4f43-a4a5-1ba3b8307f71 admin admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:50.056199 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555a85c1-4c52-4f43-a4a5-1ba3b8307f71 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:39:50.056998 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555a85c1-4c52-4f43-a4a5-1ba3b8307f71 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:39:50.058976 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555a85c1-4c52-4f43-a4a5-1ba3b8307f71 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:39:50.075897 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1084/2160] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Oct 31 12:39:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:39:50.077037 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbcf766-7dec-4321-b417-0fd729c85849 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:39:50.086914 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbcf766-7dec-4321-b417-0fd729c85849 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:39:50.092722 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4f9581-5235-4c1e-b865-24f2d028aa79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:39:50.093418 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4f9581-5235-4c1e-b865-24f2d028aa79 admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:50.093774 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4f9581-5235-4c1e-b865-24f2d028aa79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:50.094123 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4f9581-5235-4c1e-b865-24f2d028aa79 admin admin] PATH_INFO: `/v3/projects` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:50.094836 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4f9581-5235-4c1e-b865-24f2d028aa79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:39:50.096865 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4f9581-5235-4c1e-b865-24f2d028aa79 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:39:50.130245 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1077/2161] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:39:50 2022] POST /identity/v3/projects => generated 403 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 31 12:39:50.134654 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555a85c1-4c52-4f43-a4a5-1ba3b8307f71 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:39:50.148850 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555a85c1-4c52-4f43-a4a5-1ba3b8307f71 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:39:50.234668 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f7d325-9db2-4388-b652-ab368345657b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:39:50.235612 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f7d325-9db2-4388-b652-ab368345657b admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:50.236061 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f7d325-9db2-4388-b652-ab368345657b admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:50.236509 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f7d325-9db2-4388-b652-ab368345657b admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:50.237299 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f7d325-9db2-4388-b652-ab368345657b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:39:50.239561 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f7d325-9db2-4388-b652-ab368345657b admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:39:50.266030 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1085/2162] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:39:50 2022] POST /identity/v3/users => generated 302 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:39:50.270436 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4f9581-5235-4c1e-b865-24f2d028aa79 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:39:50.282369 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4f9581-5235-4c1e-b865-24f2d028aa79 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:39:50.287213 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f79a76-42d9-4c29-b449-1513460161c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:39:50.288077 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f79a76-42d9-4c29-b449-1513460161c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:50.288352 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f79a76-42d9-4c29-b449-1513460161c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:50.288622 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f79a76-42d9-4c29-b449-1513460161c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:50.289333 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f79a76-42d9-4c29-b449-1513460161c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:39:50.291676 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f79a76-42d9-4c29-b449-1513460161c4 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:39:50.305689 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1078/2163] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:39:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 31 12:39:50.307349 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f7d325-9db2-4388-b652-ab368345657b admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:39:50.319635 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f7d325-9db2-4388-b652-ab368345657b admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:39:50.325414 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a407ed1f-cd79-4eb0-83ea-b174e71c76ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:39:50.326032 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a407ed1f-cd79-4eb0-83ea-b174e71c76ed admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:50.326312 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a407ed1f-cd79-4eb0-83ea-b174e71c76ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:50.326585 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a407ed1f-cd79-4eb0-83ea-b174e71c76ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:50.327254 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a407ed1f-cd79-4eb0-83ea-b174e71c76ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:39:50.328851 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a407ed1f-cd79-4eb0-83ea-b174e71c76ed admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:39:50.340718 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1086/2164] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:39:50 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:39:50.344898 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f79a76-42d9-4c29-b449-1513460161c4 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:39:50.355784 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f79a76-42d9-4c29-b449-1513460161c4 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:39:50.361192 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb39f94-b989-447b-abf9-bd3e9aaec2ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:39:50.361851 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb39f94-b989-447b-abf9-bd3e9aaec2ff admin admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:50.362158 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb39f94-b989-447b-abf9-bd3e9aaec2ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:50.362435 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb39f94-b989-447b-abf9-bd3e9aaec2ff admin admin] PATH_INFO: `/v3/projects/e1f09c824b854b8b863fcd35b90b36e6/users/6d0ca235b0c044eca6f108d7e730dcea/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:50.379914 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb39f94-b989-447b-abf9-bd3e9aaec2ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1f09c824b854b8b863fcd35b90b36e6, user_id=6d0ca235b0c044eca6f108d7e730dcea, role_id=ca1a5695f8bd499aa0b8cedaeecd2cc6)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:39:50.382808 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb39f94-b989-447b-abf9-bd3e9aaec2ff admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:39:50.386313 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:39:50.399679 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1079/2165] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:39:50 2022] PUT /identity/v3/projects/e1f09c824b854b8b863fcd35b90b36e6/users/6d0ca235b0c044eca6f108d7e730dcea/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:39:50.411112 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac1b384-550b-4910-a9bc-2c8183388633 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:50.411430 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac1b384-550b-4910-a9bc-2c8183388633 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:50.411703 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac1b384-550b-4910-a9bc-2c8183388633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:50.441105 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-2ac1b384-550b-4910-a9bc-2c8183388633 None None] MFA Rules not processed for user `6d0ca235b0c044eca6f108d7e730dcea`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:39:50.442324 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-2ac1b384-550b-4910-a9bc-2c8183388633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:39:50.477485 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:39:50.529626 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1087/2166] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:39:50 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:39:50.530236 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb39f94-b989-447b-abf9-bd3e9aaec2ff admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:39:50.608422 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb39f94-b989-447b-abf9-bd3e9aaec2ff admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:39:50.614002 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bd1a91-6c46-4806-925f-6d58cf8a8908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:39:50.614612 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bd1a91-6c46-4806-925f-6d58cf8a8908 admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:50.614893 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bd1a91-6c46-4806-925f-6d58cf8a8908 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:50.615182 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bd1a91-6c46-4806-925f-6d58cf8a8908 admin admin] PATH_INFO: `/v3/projects` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:50.615827 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bd1a91-6c46-4806-925f-6d58cf8a8908 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:39:50.617734 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bd1a91-6c46-4806-925f-6d58cf8a8908 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:39:50.647478 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1080/2167] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:39:50 2022] POST /identity/v3/projects => generated 401 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:39:50.649597 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac1b384-550b-4910-a9bc-2c8183388633 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:39:50.661720 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac1b384-550b-4910-a9bc-2c8183388633 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:39:50.724509 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affac01b-0a1c-4436-b41b-d0dcea97fca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:39:50.724805 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affac01b-0a1c-4436-b41b-d0dcea97fca7 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:50.725209 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affac01b-0a1c-4436-b41b-d0dcea97fca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:50.725487 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affac01b-0a1c-4436-b41b-d0dcea97fca7 admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:50.726127 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affac01b-0a1c-4436-b41b-d0dcea97fca7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:39:50.730372 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affac01b-0a1c-4436-b41b-d0dcea97fca7 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:39:50.731844 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bd1a91-6c46-4806-925f-6d58cf8a8908 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:39:50.733377 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-20bd1a91-6c46-4806-925f-6d58cf8a8908 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:39:50.750381 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bd1a91-6c46-4806-925f-6d58cf8a8908 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:39:50.755328 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbccb05-7ab1-47e5-b410-dcc17df42c6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:39:50.756044 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbccb05-7ab1-47e5-b410-dcc17df42c6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:50.756493 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbccb05-7ab1-47e5-b410-dcc17df42c6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:50.756941 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbccb05-7ab1-47e5-b410-dcc17df42c6c admin admin] PATH_INFO: `/v3/users/d6c1beb321e44908ab987b17f7b74484` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:50.761293 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1088/2168] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:39:50 2022] POST /identity/v3/users => generated 307 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:39:50.761450 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbccb05-7ab1-47e5-b410-dcc17df42c6c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6c1beb321e44908ab987b17f7b74484)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:39:50.761450 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbccb05-7ab1-47e5-b410-dcc17df42c6c admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:39:50.765596 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affac01b-0a1c-4436-b41b-d0dcea97fca7 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:39:50.776223 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affac01b-0a1c-4436-b41b-d0dcea97fca7 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:39:50.781010 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c162b61e-3340-4fda-99a8-dfd1bc4e8dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:39:50.781615 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c162b61e-3340-4fda-99a8-dfd1bc4e8dce admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:50.781984 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c162b61e-3340-4fda-99a8-dfd1bc4e8dce admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:50.782233 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c162b61e-3340-4fda-99a8-dfd1bc4e8dce admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:50.782869 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c162b61e-3340-4fda-99a8-dfd1bc4e8dce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:39:50.784694 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c162b61e-3340-4fda-99a8-dfd1bc4e8dce admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:39:50.798621 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1089/2169] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:39:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:39:50.799636 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c162b61e-3340-4fda-99a8-dfd1bc4e8dce admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:39:50.809783 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c162b61e-3340-4fda-99a8-dfd1bc4e8dce admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:39:50.812913 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-6fbccb05-7ab1-47e5-b410-dcc17df42c6c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6c1beb321e44908ab987b17f7b74484'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:39:50.814388 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa9d96b-cfd6-49cb-99ac-6314f4ea3817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:39:50.815027 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa9d96b-cfd6-49cb-99ac-6314f4ea3817 admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:50.815346 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa9d96b-cfd6-49cb-99ac-6314f4ea3817 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:50.815617 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa9d96b-cfd6-49cb-99ac-6314f4ea3817 admin admin] PATH_INFO: `/v3/projects/0904b0d2263b432bbd2104acb4611377/users/9a81e5d34f0a4634a78686466a386dc8/roles/176c07b4fe3b4537bdfeb4be21b379c5` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:50.820393 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-6fbccb05-7ab1-47e5-b410-dcc17df42c6c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6c1beb321e44908ab987b17f7b74484'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:39:50.821085 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-6fbccb05-7ab1-47e5-b410-dcc17df42c6c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6c1beb321e44908ab987b17f7b74484'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:39:50.833933 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-6fbccb05-7ab1-47e5-b410-dcc17df42c6c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6c1beb321e44908ab987b17f7b74484'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:39:50.838775 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa9d96b-cfd6-49cb-99ac-6314f4ea3817 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0904b0d2263b432bbd2104acb4611377, user_id=9a81e5d34f0a4634a78686466a386dc8, role_id=176c07b4fe3b4537bdfeb4be21b379c5)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:39:50.841382 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-6fbccb05-7ab1-47e5-b410-dcc17df42c6c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6c1beb321e44908ab987b17f7b74484'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:39:50.841534 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa9d96b-cfd6-49cb-99ac-6314f4ea3817 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:39:50.847816 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1081/2170] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:39:50 2022] DELETE /identity/v3/users/d6c1beb321e44908ab987b17f7b74484 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 31 12:39:50.857045 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1090/2171] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:39:50 2022] PUT /identity/v3/projects/0904b0d2263b432bbd2104acb4611377/users/9a81e5d34f0a4634a78686466a386dc8/roles/176c07b4fe3b4537bdfeb4be21b379c5 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 31 12:39:50.861021 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbccb05-7ab1-47e5-b410-dcc17df42c6c admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:39:50.862419 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-6fbccb05-7ab1-47e5-b410-dcc17df42c6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:39:50.917538 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:39:50.932687 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbccb05-7ab1-47e5-b410-dcc17df42c6c admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:39:50.997871 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48efca90-25ce-4f64-86fa-740da9d41708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:39:50.998493 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48efca90-25ce-4f64-86fa-740da9d41708 admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:50.998765 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48efca90-25ce-4f64-86fa-740da9d41708 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:50.999032 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48efca90-25ce-4f64-86fa-740da9d41708 admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:50.999709 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48efca90-25ce-4f64-86fa-740da9d41708 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:39:51.001292 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48efca90-25ce-4f64-86fa-740da9d41708 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:39:51.011577 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1082/2172] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:39:50 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:39:51.015702 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa9d96b-cfd6-49cb-99ac-6314f4ea3817 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:39:51.029578 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48efca90-25ce-4f64-86fa-740da9d41708 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:39:51.031705 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-48efca90-25ce-4f64-86fa-740da9d41708 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:39:51.041200 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa9d96b-cfd6-49cb-99ac-6314f4ea3817 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:39:51.046784 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48efca90-25ce-4f64-86fa-740da9d41708 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:39:51.051552 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99093b45-fb18-4ab2-aecb-35c733f01a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:39:51.052179 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99093b45-fb18-4ab2-aecb-35c733f01a0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:51.052455 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99093b45-fb18-4ab2-aecb-35c733f01a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:51.052729 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99093b45-fb18-4ab2-aecb-35c733f01a0f admin admin] PATH_INFO: `/v3/projects/b9a2bf9a5a484523a700e4e6c568d4de` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:51.053870 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99093b45-fb18-4ab2-aecb-35c733f01a0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9a2bf9a5a484523a700e4e6c568d4de)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:39:51.055806 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99093b45-fb18-4ab2-aecb-35c733f01a0f admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:39:51.059937 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14f7f1c-7696-436f-9291-1f6f24764c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:39:51.064326 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14f7f1c-7696-436f-9291-1f6f24764c37 admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:51.064612 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14f7f1c-7696-436f-9291-1f6f24764c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:51.065004 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14f7f1c-7696-436f-9291-1f6f24764c37 admin admin] PATH_INFO: `/v3/projects/0904b0d2263b432bbd2104acb4611377/users/9a81e5d34f0a4634a78686466a386dc8/roles/176c07b4fe3b4537bdfeb4be21b379c5` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:51.067697 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14f7f1c-7696-436f-9291-1f6f24764c37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0904b0d2263b432bbd2104acb4611377, user_id=9a81e5d34f0a4634a78686466a386dc8, role_id=176c07b4fe3b4537bdfeb4be21b379c5)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:39:51.070391 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14f7f1c-7696-436f-9291-1f6f24764c37 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:39:51.081653 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1091/2173] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:39:51 2022] PUT /identity/v3/projects/0904b0d2263b432bbd2104acb4611377/users/9a81e5d34f0a4634a78686466a386dc8/roles/176c07b4fe3b4537bdfeb4be21b379c5 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 31 12:39:51.092143 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0059b88a-6672-4b9c-a18b-fa6f91afefe1 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:51.092429 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0059b88a-6672-4b9c-a18b-fa6f91afefe1 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:51.092707 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0059b88a-6672-4b9c-a18b-fa6f91afefe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:51.110800 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-99093b45-fb18-4ab2-aecb-35c733f01a0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9a2bf9a5a484523a700e4e6c568d4de', 'request_id': 'req-99093b45-fb18-4ab2-aecb-35c733f01a0f'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:39:51.117572 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-99093b45-fb18-4ab2-aecb-35c733f01a0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9a2bf9a5a484523a700e4e6c568d4de', 'request_id': 'req-99093b45-fb18-4ab2-aecb-35c733f01a0f'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:39:51.121069 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-0059b88a-6672-4b9c-a18b-fa6f91afefe1 None None] MFA Rules not processed for user `9a81e5d34f0a4634a78686466a386dc8`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:39:51.122295 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-0059b88a-6672-4b9c-a18b-fa6f91afefe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:39:51.125439 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-99093b45-fb18-4ab2-aecb-35c733f01a0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9a2bf9a5a484523a700e4e6c568d4de', 'request_id': 'req-99093b45-fb18-4ab2-aecb-35c733f01a0f'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:39:51.140006 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-99093b45-fb18-4ab2-aecb-35c733f01a0f admin admin] The token cache is being invalidate because project b9a2bf9a5a484523a700e4e6c568d4de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:39:51.140296 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-99093b45-fb18-4ab2-aecb-35c733f01a0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:39:51.140949 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-99093b45-fb18-4ab2-aecb-35c733f01a0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:39:51.142460 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1083/2174] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:39:51 2022] DELETE /identity/v3/projects/b9a2bf9a5a484523a700e4e6c568d4de => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 31 12:39:51.147549 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0158331a-b413-4c9e-822e-c139d866b730 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:51.147830 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0158331a-b413-4c9e-822e-c139d866b730 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:51.148490 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0158331a-b413-4c9e-822e-c139d866b730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:51.168243 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-0158331a-b413-4c9e-822e-c139d866b730 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:39:51.169432 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-0158331a-b413-4c9e-822e-c139d866b730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:39:51.188835 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:39:51.192715 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1092/2175] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:39:51 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 31 12:39:51.205196 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0059b88a-6672-4b9c-a18b-fa6f91afefe1 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:39:51.206551 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-0059b88a-6672-4b9c-a18b-fa6f91afefe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:39:51.228704 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1084/2176] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:39:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 31 12:39:51.237831 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec61fd70-4571-48e1-883d-70fed824b354 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:51.238117 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec61fd70-4571-48e1-883d-70fed824b354 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:51.238386 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec61fd70-4571-48e1-883d-70fed824b354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:51.265284 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-ec61fd70-4571-48e1-883d-70fed824b354 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:39:51.266477 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-ec61fd70-4571-48e1-883d-70fed824b354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:39:51.271674 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0059b88a-6672-4b9c-a18b-fa6f91afefe1 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:39:51.274543 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1085/2177] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:39:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 31 12:39:51.276277 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06c4c7b-367c-45a6-98f2-9316d2fd2468 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d6d7ea077d49e8a66c6c6762499702', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:39:51.276903 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06c4c7b-367c-45a6-98f2-9316d2fd2468 service cinder] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:51.277185 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06c4c7b-367c-45a6-98f2-9316d2fd2468 service cinder] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:51.277456 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06c4c7b-367c-45a6-98f2-9316d2fd2468 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:51.279111 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec61fd70-4571-48e1-883d-70fed824b354 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:39:51.286623 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06c4c7b-367c-45a6-98f2-9316d2fd2468 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:39:51.288201 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06c4c7b-367c-45a6-98f2-9316d2fd2468 service cinder] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:39:51.288339 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec61fd70-4571-48e1-883d-70fed824b354 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:39:51.293038 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8c782d-9d00-40c4-b904-2efb13a23422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:39:51.293620 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8c782d-9d00-40c4-b904-2efb13a23422 admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:51.293893 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8c782d-9d00-40c4-b904-2efb13a23422 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:51.294158 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8c782d-9d00-40c4-b904-2efb13a23422 admin admin] PATH_INFO: `/v3/domains` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:51.294785 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8c782d-9d00-40c4-b904-2efb13a23422 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:39:51.295082 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8c782d-9d00-40c4-b904-2efb13a23422 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:39:51.296677 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8c782d-9d00-40c4-b904-2efb13a23422 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:39:51.297880 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1093/2178] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:39:51 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:39:51.305266 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:39:51.310956 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1086/2179] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Oct 31 12:39:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 31 12:39:51.314155 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06c4c7b-367c-45a6-98f2-9316d2fd2468 service cinder] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:39:51.324498 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06c4c7b-367c-45a6-98f2-9316d2fd2468 service cinder] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:39:51.329459 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd1477d-d415-4d2c-92df-904f59c04cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:39:51.329953 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd1477d-d415-4d2c-92df-904f59c04cca admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:51.330229 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd1477d-d415-4d2c-92df-904f59c04cca admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:51.330497 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd1477d-d415-4d2c-92df-904f59c04cca admin admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:51.331135 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd1477d-d415-4d2c-92df-904f59c04cca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:39:51.333085 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd1477d-d415-4d2c-92df-904f59c04cca admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:39:51.363621 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1094/2180] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:39:51 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:39:51.366781 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8c782d-9d00-40c4-b904-2efb13a23422 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:39:51.377675 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8c782d-9d00-40c4-b904-2efb13a23422 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:39:51.437888 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd1477d-d415-4d2c-92df-904f59c04cca admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:39:51.439819 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-9cd1477d-d415-4d2c-92df-904f59c04cca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:39:51.440238 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb455a0-de28-4ca6-bf6a-294cc2df42bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:39:51.440902 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb455a0-de28-4ca6-bf6a-294cc2df42bd admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:51.441178 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb455a0-de28-4ca6-bf6a-294cc2df42bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:51.441446 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb455a0-de28-4ca6-bf6a-294cc2df42bd admin admin] PATH_INFO: `/v3/users` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:51.442066 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb455a0-de28-4ca6-bf6a-294cc2df42bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:39:51.444063 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb455a0-de28-4ca6-bf6a-294cc2df42bd admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:39:51.452517 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd1477d-d415-4d2c-92df-904f59c04cca admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:39:51.457122 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387f6f5b-dd2d-48be-b333-45b113fb09a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:39:51.457742 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387f6f5b-dd2d-48be-b333-45b113fb09a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:51.458055 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387f6f5b-dd2d-48be-b333-45b113fb09a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:51.458282 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387f6f5b-dd2d-48be-b333-45b113fb09a9 admin admin] PATH_INFO: `/v3/users/6d0ca235b0c044eca6f108d7e730dcea` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:51.459421 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387f6f5b-dd2d-48be-b333-45b113fb09a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d0ca235b0c044eca6f108d7e730dcea)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:39:51.461953 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387f6f5b-dd2d-48be-b333-45b113fb09a9 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:39:51.466126 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1087/2181] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:39:51 2022] POST /identity/v3/users => generated 287 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:39:51.469979 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb455a0-de28-4ca6-bf6a-294cc2df42bd admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:39:51.480014 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb455a0-de28-4ca6-bf6a-294cc2df42bd admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:39:51.485537 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c566a3-4462-4740-a272-db6814473b84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:39:51.486123 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c566a3-4462-4740-a272-db6814473b84 admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:51.486399 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c566a3-4462-4740-a272-db6814473b84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:51.486667 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c566a3-4462-4740-a272-db6814473b84 admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:51.487821 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c566a3-4462-4740-a272-db6814473b84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:39:51.490061 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c566a3-4462-4740-a272-db6814473b84 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:39:51.503200 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1088/2182] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:39:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 31 12:39:51.506898 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c566a3-4462-4740-a272-db6814473b84 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:39:51.517002 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c566a3-4462-4740-a272-db6814473b84 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:39:51.518684 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-387f6f5b-dd2d-48be-b333-45b113fb09a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d0ca235b0c044eca6f108d7e730dcea'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:39:51.522570 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f5e92c-1bb8-4c42-a3ad-6879f0618bde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:39:51.523279 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f5e92c-1bb8-4c42-a3ad-6879f0618bde admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:51.523460 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f5e92c-1bb8-4c42-a3ad-6879f0618bde admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:51.523767 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f5e92c-1bb8-4c42-a3ad-6879f0618bde admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:51.524386 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f5e92c-1bb8-4c42-a3ad-6879f0618bde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:39:51.526481 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f5e92c-1bb8-4c42-a3ad-6879f0618bde admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:39:51.528005 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-387f6f5b-dd2d-48be-b333-45b113fb09a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d0ca235b0c044eca6f108d7e730dcea'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:39:51.528661 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-387f6f5b-dd2d-48be-b333-45b113fb09a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d0ca235b0c044eca6f108d7e730dcea'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:39:51.529309 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-387f6f5b-dd2d-48be-b333-45b113fb09a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d0ca235b0c044eca6f108d7e730dcea'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:39:51.537432 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1089/2183] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:39:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 31 12:39:51.542138 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f5e92c-1bb8-4c42-a3ad-6879f0618bde admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:39:51.544228 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-a5f5e92c-1bb8-4c42-a3ad-6879f0618bde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:39:51.550330 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-387f6f5b-dd2d-48be-b333-45b113fb09a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d0ca235b0c044eca6f108d7e730dcea'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:39:51.557736 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f5e92c-1bb8-4c42-a3ad-6879f0618bde admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:39:51.562550 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0ccffc-054b-4573-af66-080eb56445da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:39:51.563152 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0ccffc-054b-4573-af66-080eb56445da admin admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:51.563459 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0ccffc-054b-4573-af66-080eb56445da admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:51.563733 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0ccffc-054b-4573-af66-080eb56445da admin admin] PATH_INFO: `/v3/projects/8686fe87fb534b65a0f5cf77dbbc18a5/users/3ef96b7cf1274462b8a82d814fdb1043/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:51.571606 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1095/2184] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:39:51 2022] DELETE /identity/v3/users/6d0ca235b0c044eca6f108d7e730dcea => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 31 12:39:51.576472 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3b13dc-694b-4daa-b2ec-35d5c18e88a9 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:51.576763 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3b13dc-694b-4daa-b2ec-35d5c18e88a9 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:51.577038 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3b13dc-694b-4daa-b2ec-35d5c18e88a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:51.589741 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0ccffc-054b-4573-af66-080eb56445da admin admin] RBAC: Authorizing `identity:create_grant(project_id=8686fe87fb534b65a0f5cf77dbbc18a5, user_id=3ef96b7cf1274462b8a82d814fdb1043, role_id=ca1a5695f8bd499aa0b8cedaeecd2cc6)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:39:51.592746 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0ccffc-054b-4573-af66-080eb56445da admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:39:51.601913 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-8a3b13dc-694b-4daa-b2ec-35d5c18e88a9 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:39:51.603928 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-8a3b13dc-694b-4daa-b2ec-35d5c18e88a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:39:51.610926 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1090/2185] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:39:51 2022] PUT /identity/v3/projects/8686fe87fb534b65a0f5cf77dbbc18a5/users/3ef96b7cf1274462b8a82d814fdb1043/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 31 12:39:51.623463 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4942aea1-1e44-4454-9e51-d65f102fe76e None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:51.623664 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4942aea1-1e44-4454-9e51-d65f102fe76e None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:51.623931 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4942aea1-1e44-4454-9e51-d65f102fe76e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:51.654028 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-4942aea1-1e44-4454-9e51-d65f102fe76e None None] MFA Rules not processed for user `3ef96b7cf1274462b8a82d814fdb1043`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:39:51.655274 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-4942aea1-1e44-4454-9e51-d65f102fe76e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:39:51.667790 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:39:51.682759 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1096/2186] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:39:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 31 12:39:51.692442 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3b13dc-694b-4daa-b2ec-35d5c18e88a9 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:39:51.693779 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-8a3b13dc-694b-4daa-b2ec-35d5c18e88a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:39:51.723053 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1091/2187] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:39:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 31 12:39:51.735291 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4942aea1-1e44-4454-9e51-d65f102fe76e None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:39:51.736642 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-4942aea1-1e44-4454-9e51-d65f102fe76e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:39:51.756152 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3b13dc-694b-4daa-b2ec-35d5c18e88a9 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:39:51.760038 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a031e3dd-0fcc-4fde-9fa5-ce9ff81c87e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:39:51.760748 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a031e3dd-0fcc-4fde-9fa5-ce9ff81c87e1 service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:51.761121 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a031e3dd-0fcc-4fde-9fa5-ce9ff81c87e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:51.761506 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a031e3dd-0fcc-4fde-9fa5-ce9ff81c87e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:51.770699 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a031e3dd-0fcc-4fde-9fa5-ce9ff81c87e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:39:51.772218 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a031e3dd-0fcc-4fde-9fa5-ce9ff81c87e1 service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:39:51.773071 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:39:51.782265 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1097/2188] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:39:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 31 12:39:51.813789 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4942aea1-1e44-4454-9e51-d65f102fe76e None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:39:51.819379 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c04b26a-5cca-43f0-9820-a062b5d2f724 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99889b1dab0b471d9c77887cffefc652', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:39:51.819994 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c04b26a-5cca-43f0-9820-a062b5d2f724 service nova] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:51.820271 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c04b26a-5cca-43f0-9820-a062b5d2f724 service nova] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:51.820548 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c04b26a-5cca-43f0-9820-a062b5d2f724 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:51.828974 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c04b26a-5cca-43f0-9820-a062b5d2f724 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:39:51.830541 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c04b26a-5cca-43f0-9820-a062b5d2f724 service nova] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:39:51.840350 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1092/2189] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:39:51 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 31 12:39:52.074044 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a031e3dd-0fcc-4fde-9fa5-ce9ff81c87e1 service neutron] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:39:52.083804 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c04b26a-5cca-43f0-9820-a062b5d2f724 service nova] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:39:52.084716 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a031e3dd-0fcc-4fde-9fa5-ce9ff81c87e1 service neutron] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:39:52.085530 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-8c04b26a-5cca-43f0-9820-a062b5d2f724 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:39:52.089800 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444e72a7-edfc-4d06-9efc-407050d001b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:39:52.090467 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444e72a7-edfc-4d06-9efc-407050d001b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:52.090744 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444e72a7-edfc-4d06-9efc-407050d001b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:52.091019 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444e72a7-edfc-4d06-9efc-407050d001b0 admin admin] PATH_INFO: `/v3/users/3ef96b7cf1274462b8a82d814fdb1043` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:52.092185 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444e72a7-edfc-4d06-9efc-407050d001b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ef96b7cf1274462b8a82d814fdb1043)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:39:52.094090 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444e72a7-edfc-4d06-9efc-407050d001b0 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:39:52.107385 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c04b26a-5cca-43f0-9820-a062b5d2f724 service nova] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:39:52.112907 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdaab60-6d1d-46db-b89f-002d85aba70b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:39:52.113830 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdaab60-6d1d-46db-b89f-002d85aba70b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:52.114364 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdaab60-6d1d-46db-b89f-002d85aba70b admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:52.114802 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdaab60-6d1d-46db-b89f-002d85aba70b admin admin] PATH_INFO: `/v3/projects/e1f09c824b854b8b863fcd35b90b36e6` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:52.116005 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdaab60-6d1d-46db-b89f-002d85aba70b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1f09c824b854b8b863fcd35b90b36e6)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:39:52.117879 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdaab60-6d1d-46db-b89f-002d85aba70b admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:39:52.149696 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-444e72a7-edfc-4d06-9efc-407050d001b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ef96b7cf1274462b8a82d814fdb1043'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:39:52.160271 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-444e72a7-edfc-4d06-9efc-407050d001b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ef96b7cf1274462b8a82d814fdb1043'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:39:52.160936 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-444e72a7-edfc-4d06-9efc-407050d001b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ef96b7cf1274462b8a82d814fdb1043'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:39:52.161644 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-444e72a7-edfc-4d06-9efc-407050d001b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ef96b7cf1274462b8a82d814fdb1043'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:39:52.175043 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-444e72a7-edfc-4d06-9efc-407050d001b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ef96b7cf1274462b8a82d814fdb1043'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:39:52.179888 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-9cdaab60-6d1d-46db-b89f-002d85aba70b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1f09c824b854b8b863fcd35b90b36e6', 'request_id': 'req-9cdaab60-6d1d-46db-b89f-002d85aba70b'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:39:52.186295 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-9cdaab60-6d1d-46db-b89f-002d85aba70b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1f09c824b854b8b863fcd35b90b36e6', 'request_id': 'req-9cdaab60-6d1d-46db-b89f-002d85aba70b'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:39:52.187985 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1098/2190] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:39:52 2022] DELETE /identity/v3/users/3ef96b7cf1274462b8a82d814fdb1043 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 31 12:39:52.192863 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa469a9-6a1f-4b81-a496-2bf848181bed None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:52.193243 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa469a9-6a1f-4b81-a496-2bf848181bed None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:52.193544 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa469a9-6a1f-4b81-a496-2bf848181bed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:52.194756 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-9cdaab60-6d1d-46db-b89f-002d85aba70b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1f09c824b854b8b863fcd35b90b36e6', 'request_id': 'req-9cdaab60-6d1d-46db-b89f-002d85aba70b'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:39:52.206678 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-9cdaab60-6d1d-46db-b89f-002d85aba70b admin admin] The token cache is being invalidate because project e1f09c824b854b8b863fcd35b90b36e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:39:52.206962 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-9cdaab60-6d1d-46db-b89f-002d85aba70b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:39:52.207619 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-9cdaab60-6d1d-46db-b89f-002d85aba70b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:39:52.209119 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1093/2191] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:39:52 2022] DELETE /identity/v3/projects/e1f09c824b854b8b863fcd35b90b36e6 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 31 12:39:52.213573 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdaab60-6d1d-46db-b89f-002d85aba70b admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:39:52.214866 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-9cdaab60-6d1d-46db-b89f-002d85aba70b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:39:52.217872 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-2fa469a9-6a1f-4b81-a496-2bf848181bed None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:39:52.219098 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-2fa469a9-6a1f-4b81-a496-2bf848181bed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:39:52.282425 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdaab60-6d1d-46db-b89f-002d85aba70b admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:39:52.284082 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1099/2192] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:39:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:39:52.287712 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40b281f-34b3-41cf-9fb7-22eb83ea4660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:39:52.287956 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40b281f-34b3-41cf-9fb7-22eb83ea4660 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:52.288205 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40b281f-34b3-41cf-9fb7-22eb83ea4660 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:52.288472 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40b281f-34b3-41cf-9fb7-22eb83ea4660 admin admin] PATH_INFO: `/v3/users/9a81e5d34f0a4634a78686466a386dc8` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:52.289578 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40b281f-34b3-41cf-9fb7-22eb83ea4660 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a81e5d34f0a4634a78686466a386dc8)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:39:52.292157 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40b281f-34b3-41cf-9fb7-22eb83ea4660 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:39:52.295081 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa469a9-6a1f-4b81-a496-2bf848181bed None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:39:52.296505 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-2fa469a9-6a1f-4b81-a496-2bf848181bed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:39:52.307731 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:39:52.339794 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:39:52.342845 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-b40b281f-34b3-41cf-9fb7-22eb83ea4660 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a81e5d34f0a4634a78686466a386dc8'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:39:52.351588 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-b40b281f-34b3-41cf-9fb7-22eb83ea4660 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a81e5d34f0a4634a78686466a386dc8'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:39:52.352409 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-b40b281f-34b3-41cf-9fb7-22eb83ea4660 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a81e5d34f0a4634a78686466a386dc8'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:39:52.360645 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa469a9-6a1f-4b81-a496-2bf848181bed None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:39:52.364308 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7306eb7-b2e3-4bf5-b568-3a16e0a29817 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:39:52.364914 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7306eb7-b2e3-4bf5-b568-3a16e0a29817 service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:52.365192 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7306eb7-b2e3-4bf5-b568-3a16e0a29817 service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:52.365408 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-b40b281f-34b3-41cf-9fb7-22eb83ea4660 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a81e5d34f0a4634a78686466a386dc8'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:39:52.365544 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7306eb7-b2e3-4bf5-b568-3a16e0a29817 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:52.372613 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-b40b281f-34b3-41cf-9fb7-22eb83ea4660 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a81e5d34f0a4634a78686466a386dc8'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:39:52.374672 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7306eb7-b2e3-4bf5-b568-3a16e0a29817 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:39:52.376155 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7306eb7-b2e3-4bf5-b568-3a16e0a29817 service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:39:52.378461 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1094/2193] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:39:52 2022] DELETE /identity/v3/users/9a81e5d34f0a4634a78686466a386dc8 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 31 12:39:52.388362 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1100/2194] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:39:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 31 12:39:52.767778 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40b281f-34b3-41cf-9fb7-22eb83ea4660 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:39:52.770063 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-b40b281f-34b3-41cf-9fb7-22eb83ea4660 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:39:52.810590 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7306eb7-b2e3-4bf5-b568-3a16e0a29817 service neutron] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:39:52.812244 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-a7306eb7-b2e3-4bf5-b568-3a16e0a29817 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:39:52.826935 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7306eb7-b2e3-4bf5-b568-3a16e0a29817 service neutron] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:39:52.861509 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40b281f-34b3-41cf-9fb7-22eb83ea4660 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:39:52.866197 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab90c14-38aa-4361-8afb-ff60862ac43d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:39:52.866819 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab90c14-38aa-4361-8afb-ff60862ac43d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:52.867098 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab90c14-38aa-4361-8afb-ff60862ac43d admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:52.867402 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab90c14-38aa-4361-8afb-ff60862ac43d admin admin] PATH_INFO: `/v3/projects/0904b0d2263b432bbd2104acb4611377` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:52.868522 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab90c14-38aa-4361-8afb-ff60862ac43d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0904b0d2263b432bbd2104acb4611377)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:39:52.871112 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab90c14-38aa-4361-8afb-ff60862ac43d admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:39:52.888283 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be753988-d7f9-43eb-a111-f5ac49411e9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:39:52.888924 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be753988-d7f9-43eb-a111-f5ac49411e9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:52.889200 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be753988-d7f9-43eb-a111-f5ac49411e9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:52.889470 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be753988-d7f9-43eb-a111-f5ac49411e9a admin admin] PATH_INFO: `/v3/projects/8686fe87fb534b65a0f5cf77dbbc18a5` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:52.890675 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be753988-d7f9-43eb-a111-f5ac49411e9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=8686fe87fb534b65a0f5cf77dbbc18a5)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:39:52.892657 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be753988-d7f9-43eb-a111-f5ac49411e9a admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:39:52.920497 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-2ab90c14-38aa-4361-8afb-ff60862ac43d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0904b0d2263b432bbd2104acb4611377', 'request_id': 'req-2ab90c14-38aa-4361-8afb-ff60862ac43d'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:39:52.926562 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-2ab90c14-38aa-4361-8afb-ff60862ac43d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0904b0d2263b432bbd2104acb4611377', 'request_id': 'req-2ab90c14-38aa-4361-8afb-ff60862ac43d'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:39:52.935302 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-2ab90c14-38aa-4361-8afb-ff60862ac43d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0904b0d2263b432bbd2104acb4611377', 'request_id': 'req-2ab90c14-38aa-4361-8afb-ff60862ac43d'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:39:52.942292 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-be753988-d7f9-43eb-a111-f5ac49411e9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8686fe87fb534b65a0f5cf77dbbc18a5', 'request_id': 'req-be753988-d7f9-43eb-a111-f5ac49411e9a'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:39:52.946739 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-2ab90c14-38aa-4361-8afb-ff60862ac43d admin admin] The token cache is being invalidate because project 0904b0d2263b432bbd2104acb4611377 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:39:52.947743 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-2ab90c14-38aa-4361-8afb-ff60862ac43d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:39:52.948468 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-2ab90c14-38aa-4361-8afb-ff60862ac43d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:39:52.949596 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-be753988-d7f9-43eb-a111-f5ac49411e9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8686fe87fb534b65a0f5cf77dbbc18a5', 'request_id': 'req-be753988-d7f9-43eb-a111-f5ac49411e9a'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:39:52.949963 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1095/2195] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:39:52 2022] DELETE /identity/v3/projects/0904b0d2263b432bbd2104acb4611377 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 31 12:39:52.954856 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a588e4d5-418e-493d-abf1-625d1e668f8a None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:52.955201 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a588e4d5-418e-493d-abf1-625d1e668f8a None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:52.955512 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a588e4d5-418e-493d-abf1-625d1e668f8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:52.957252 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-be753988-d7f9-43eb-a111-f5ac49411e9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8686fe87fb534b65a0f5cf77dbbc18a5', 'request_id': 'req-be753988-d7f9-43eb-a111-f5ac49411e9a'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:39:52.969371 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-be753988-d7f9-43eb-a111-f5ac49411e9a admin admin] The token cache is being invalidate because project 8686fe87fb534b65a0f5cf77dbbc18a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:39:52.970097 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-be753988-d7f9-43eb-a111-f5ac49411e9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:39:52.970785 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-be753988-d7f9-43eb-a111-f5ac49411e9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:39:52.972323 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1101/2196] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:39:52 2022] DELETE /identity/v3/projects/8686fe87fb534b65a0f5cf77dbbc18a5 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 31 12:39:52.975007 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-a588e4d5-418e-493d-abf1-625d1e668f8a None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:39:52.976330 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-a588e4d5-418e-493d-abf1-625d1e668f8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:39:52.976937 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60377047-22f2-400b-9661-2b3e7a891476 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:52.977247 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60377047-22f2-400b-9661-2b3e7a891476 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:52.977635 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60377047-22f2-400b-9661-2b3e7a891476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:52.997277 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-60377047-22f2-400b-9661-2b3e7a891476 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:39:52.998596 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-60377047-22f2-400b-9661-2b3e7a891476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:39:53.031535 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:39:53.042527 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:39:53.049730 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1096/2197] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:39:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 31 12:39:53.058002 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756d4bff-d575-445c-a337-88fdb4f64f4d None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:53.058283 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756d4bff-d575-445c-a337-88fdb4f64f4d None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:53.058555 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756d4bff-d575-445c-a337-88fdb4f64f4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:53.065537 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1102/2198] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:39:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 31 12:39:53.073283 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2d4522-2562-4736-9a26-4acc68237190 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:53.073572 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2d4522-2562-4736-9a26-4acc68237190 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:53.073843 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2d4522-2562-4736-9a26-4acc68237190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:53.079730 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-756d4bff-d575-445c-a337-88fdb4f64f4d None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:39:53.080925 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-756d4bff-d575-445c-a337-88fdb4f64f4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:39:53.089525 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1097/2199] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:39:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 31 12:39:53.093547 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756d4bff-d575-445c-a337-88fdb4f64f4d None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:39:53.093681 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-2a2d4522-2562-4736-9a26-4acc68237190 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:39:53.095464 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-2a2d4522-2562-4736-9a26-4acc68237190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:39:53.102647 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756d4bff-d575-445c-a337-88fdb4f64f4d None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:39:53.103633 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1103/2200] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:39:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 31 12:39:53.107642 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c91bf9-c7b7-42ed-a6a1-9577ea06ba74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:39:53.107803 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2d4522-2562-4736-9a26-4acc68237190 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:39:53.108228 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c91bf9-c7b7-42ed-a6a1-9577ea06ba74 admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:53.108508 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c91bf9-c7b7-42ed-a6a1-9577ea06ba74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:53.108783 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c91bf9-c7b7-42ed-a6a1-9577ea06ba74 admin admin] PATH_INFO: `/v3/domains` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:53.109419 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c91bf9-c7b7-42ed-a6a1-9577ea06ba74 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:39:53.109721 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c91bf9-c7b7-42ed-a6a1-9577ea06ba74 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:39:53.111358 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c91bf9-c7b7-42ed-a6a1-9577ea06ba74 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:39:53.117030 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2d4522-2562-4736-9a26-4acc68237190 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:39:53.122183 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7c4bb9-3ccd-4733-8f3b-982ad49c130c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:39:53.122772 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7c4bb9-3ccd-4733-8f3b-982ad49c130c admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:53.123061 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7c4bb9-3ccd-4733-8f3b-982ad49c130c admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:53.123365 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7c4bb9-3ccd-4733-8f3b-982ad49c130c admin admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:53.127028 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1098/2201] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Oct 31 12:39:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:39:53.127207 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7c4bb9-3ccd-4733-8f3b-982ad49c130c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:39:53.127207 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7c4bb9-3ccd-4733-8f3b-982ad49c130c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:39:53.127207 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7c4bb9-3ccd-4733-8f3b-982ad49c130c admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:39:53.128119 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c91bf9-c7b7-42ed-a6a1-9577ea06ba74 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:39:53.138524 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1104/2202] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Oct 31 12:39:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 31 12:39:53.138684 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c91bf9-c7b7-42ed-a6a1-9577ea06ba74 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:39:53.142004 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7c4bb9-3ccd-4733-8f3b-982ad49c130c admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:39:53.143105 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbea602-8d24-49e1-9ec1-49167e461f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:39:53.144105 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbea602-8d24-49e1-9ec1-49167e461f26 admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:53.144383 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbea602-8d24-49e1-9ec1-49167e461f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:53.144651 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbea602-8d24-49e1-9ec1-49167e461f26 admin admin] PATH_INFO: `/v3/projects` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:53.145292 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbea602-8d24-49e1-9ec1-49167e461f26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:39:53.147396 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbea602-8d24-49e1-9ec1-49167e461f26 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:39:53.152506 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7c4bb9-3ccd-4733-8f3b-982ad49c130c admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:39:53.157091 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf28cf57-b294-47ed-9f33-9139ed23e578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:39:53.157733 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf28cf57-b294-47ed-9f33-9139ed23e578 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:53.158012 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf28cf57-b294-47ed-9f33-9139ed23e578 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:53.158312 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf28cf57-b294-47ed-9f33-9139ed23e578 admin admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:53.158954 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf28cf57-b294-47ed-9f33-9139ed23e578 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:39:53.160908 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf28cf57-b294-47ed-9f33-9139ed23e578 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:39:53.178476 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1099/2203] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:39:53 2022] POST /identity/v3/projects => generated 381 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:39:53.179603 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbea602-8d24-49e1-9ec1-49167e461f26 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:39:53.189635 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1105/2204] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:39:53 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 31 12:39:53.193609 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf28cf57-b294-47ed-9f33-9139ed23e578 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:39:53.244736 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbea602-8d24-49e1-9ec1-49167e461f26 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:39:53.254803 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf28cf57-b294-47ed-9f33-9139ed23e578 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:39:53.259494 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057fed38-3e68-4852-bb42-5b4ce7e3ab27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:39:53.260302 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057fed38-3e68-4852-bb42-5b4ce7e3ab27 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:53.260582 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057fed38-3e68-4852-bb42-5b4ce7e3ab27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:53.260854 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057fed38-3e68-4852-bb42-5b4ce7e3ab27 admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:53.261477 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057fed38-3e68-4852-bb42-5b4ce7e3ab27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:39:53.263398 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057fed38-3e68-4852-bb42-5b4ce7e3ab27 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:39:53.284952 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1106/2205] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:39:53 2022] POST /identity/v3/users => generated 287 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 31 12:39:53.288906 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057fed38-3e68-4852-bb42-5b4ce7e3ab27 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:39:53.298937 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057fed38-3e68-4852-bb42-5b4ce7e3ab27 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:39:53.303264 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c336ba12-f013-462b-a6a1-07ee921dec62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:39:53.303923 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c336ba12-f013-462b-a6a1-07ee921dec62 admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:53.304169 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e7359d-7f9c-4867-8fa9-cb124a28b6a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:39:53.304293 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c336ba12-f013-462b-a6a1-07ee921dec62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:53.304460 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c336ba12-f013-462b-a6a1-07ee921dec62 admin admin] PATH_INFO: `/v3/users` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:53.304750 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e7359d-7f9c-4867-8fa9-cb124a28b6a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:53.305025 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e7359d-7f9c-4867-8fa9-cb124a28b6a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:53.305122 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c336ba12-f013-462b-a6a1-07ee921dec62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:39:53.305294 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e7359d-7f9c-4867-8fa9-cb124a28b6a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:53.305951 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e7359d-7f9c-4867-8fa9-cb124a28b6a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:39:53.307651 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c336ba12-f013-462b-a6a1-07ee921dec62 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:39:53.308270 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e7359d-7f9c-4867-8fa9-cb124a28b6a8 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:39:53.320964 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1107/2206] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:39:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 31 12:39:53.322137 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e7359d-7f9c-4867-8fa9-cb124a28b6a8 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:39:53.331413 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1100/2207] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:39:53 2022] POST /identity/v3/users => generated 291 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 31 12:39:53.331809 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e7359d-7f9c-4867-8fa9-cb124a28b6a8 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:39:53.335087 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c336ba12-f013-462b-a6a1-07ee921dec62 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:39:53.336538 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff42857c-d0e3-4861-ac03-baa511bb120e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:39:53.337195 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff42857c-d0e3-4861-ac03-baa511bb120e admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:53.337531 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff42857c-d0e3-4861-ac03-baa511bb120e admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:53.337926 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff42857c-d0e3-4861-ac03-baa511bb120e admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:53.338554 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff42857c-d0e3-4861-ac03-baa511bb120e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:39:53.340148 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff42857c-d0e3-4861-ac03-baa511bb120e admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:39:53.345898 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c336ba12-f013-462b-a6a1-07ee921dec62 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:39:53.354101 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1108/2208] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:39:53 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 31 12:39:53.354273 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e304ec33-9862-4339-af63-026e6732f4ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:39:53.354273 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e304ec33-9862-4339-af63-026e6732f4ad admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:53.354273 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e304ec33-9862-4339-af63-026e6732f4ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:53.354273 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e304ec33-9862-4339-af63-026e6732f4ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:53.354722 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e304ec33-9862-4339-af63-026e6732f4ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:39:53.354722 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e304ec33-9862-4339-af63-026e6732f4ad admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:39:53.355963 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff42857c-d0e3-4861-ac03-baa511bb120e admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:39:53.367643 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1101/2209] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:39:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 31 12:39:53.367812 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff42857c-d0e3-4861-ac03-baa511bb120e admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:39:53.368886 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e304ec33-9862-4339-af63-026e6732f4ad admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:39:53.370129 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f209ef14-60ef-4b39-8cab-5420de241b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:39:53.370769 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f209ef14-60ef-4b39-8cab-5420de241b5d admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:53.371041 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f209ef14-60ef-4b39-8cab-5420de241b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:53.371346 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f209ef14-60ef-4b39-8cab-5420de241b5d admin admin] PATH_INFO: `/v3/projects/14a4c39daaf446858c5ae85a5c5666fb/users/afbb3481efdc4d149cd65de1ba7af484/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:53.380362 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e304ec33-9862-4339-af63-026e6732f4ad admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:39:53.385875 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeee4b0-a658-42db-b88d-f2c7a82ac118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:39:53.386162 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f209ef14-60ef-4b39-8cab-5420de241b5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=14a4c39daaf446858c5ae85a5c5666fb, user_id=afbb3481efdc4d149cd65de1ba7af484, role_id=ca1a5695f8bd499aa0b8cedaeecd2cc6)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:39:53.386466 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeee4b0-a658-42db-b88d-f2c7a82ac118 admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:53.386775 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeee4b0-a658-42db-b88d-f2c7a82ac118 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:53.387048 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeee4b0-a658-42db-b88d-f2c7a82ac118 admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:53.387718 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeee4b0-a658-42db-b88d-f2c7a82ac118 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:39:53.388802 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f209ef14-60ef-4b39-8cab-5420de241b5d admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:39:53.389262 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeee4b0-a658-42db-b88d-f2c7a82ac118 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:39:53.403857 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1102/2210] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:39:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:39:53.404078 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1109/2211] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:39:53 2022] PUT /identity/v3/projects/14a4c39daaf446858c5ae85a5c5666fb/users/afbb3481efdc4d149cd65de1ba7af484/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 31 12:39:53.405275 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeee4b0-a658-42db-b88d-f2c7a82ac118 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:39:53.413114 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fa318b-6a2c-47d9-b480-df2d3dcf4619 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:53.413488 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fa318b-6a2c-47d9-b480-df2d3dcf4619 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:53.413819 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fa318b-6a2c-47d9-b480-df2d3dcf4619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:53.416607 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeee4b0-a658-42db-b88d-f2c7a82ac118 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:39:53.437666 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:39:53.444829 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-d9fa318b-6a2c-47d9-b480-df2d3dcf4619 None None] MFA Rules not processed for user `afbb3481efdc4d149cd65de1ba7af484`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:39:53.445935 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-d9fa318b-6a2c-47d9-b480-df2d3dcf4619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:39:53.477025 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678091f8-dbec-4081-ad0b-733013444754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:39:53.477662 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678091f8-dbec-4081-ad0b-733013444754 admin admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:53.477938 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678091f8-dbec-4081-ad0b-733013444754 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:53.478750 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678091f8-dbec-4081-ad0b-733013444754 admin admin] PATH_INFO: `/v3/projects/18ce8d2885af4573ae11a903c4be8a1f/users/16274992f3ec49cfbf2e939ebdaf5241/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:53.478934 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:39:53.493978 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678091f8-dbec-4081-ad0b-733013444754 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18ce8d2885af4573ae11a903c4be8a1f, user_id=16274992f3ec49cfbf2e939ebdaf5241, role_id=ca1a5695f8bd499aa0b8cedaeecd2cc6)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:39:53.496841 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678091f8-dbec-4081-ad0b-733013444754 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:39:53.510499 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1103/2212] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:39:53 2022] PUT /identity/v3/projects/18ce8d2885af4573ae11a903c4be8a1f/users/16274992f3ec49cfbf2e939ebdaf5241/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 31 12:39:53.519214 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1110/2213] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:39:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 31 12:39:53.521957 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ced86b-ca66-4525-a6f5-b6e25479701a None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:53.522243 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ced86b-ca66-4525-a6f5-b6e25479701a None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:53.522584 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ced86b-ca66-4525-a6f5-b6e25479701a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:53.523430 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fa318b-6a2c-47d9-b480-df2d3dcf4619 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:39:53.534160 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fa318b-6a2c-47d9-b480-df2d3dcf4619 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:39:53.550643 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-84ced86b-ca66-4525-a6f5-b6e25479701a None None] MFA Rules not processed for user `16274992f3ec49cfbf2e939ebdaf5241`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:39:53.551880 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-84ced86b-ca66-4525-a6f5-b6e25479701a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:39:53.592340 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012e9901-9b17-482c-9749-2c541d885d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:39:53.592953 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012e9901-9b17-482c-9749-2c541d885d72 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:53.593226 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012e9901-9b17-482c-9749-2c541d885d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:53.593546 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012e9901-9b17-482c-9749-2c541d885d72 admin admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:53.594170 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012e9901-9b17-482c-9749-2c541d885d72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:39:53.596090 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012e9901-9b17-482c-9749-2c541d885d72 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:39:53.617535 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1104/2214] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:39:53 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:39:53.621890 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ced86b-ca66-4525-a6f5-b6e25479701a None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:39:53.624351 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1111/2215] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:39:53 2022] POST /identity/v3/projects => generated 373 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:39:53.628037 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012e9901-9b17-482c-9749-2c541d885d72 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:39:53.631976 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ced86b-ca66-4525-a6f5-b6e25479701a None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:39:53.694646 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f72f61-0a42-4722-a90d-fa15c45bc848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:39:53.695155 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f72f61-0a42-4722-a90d-fa15c45bc848 admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:53.695463 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f72f61-0a42-4722-a90d-fa15c45bc848 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:53.695732 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f72f61-0a42-4722-a90d-fa15c45bc848 admin admin] PATH_INFO: `/v3/projects` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:53.696376 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f72f61-0a42-4722-a90d-fa15c45bc848 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:39:53.698720 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012e9901-9b17-482c-9749-2c541d885d72 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:39:53.698860 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f72f61-0a42-4722-a90d-fa15c45bc848 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:39:53.703446 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506f84a6-d10d-4f33-8cf3-b8d9ba14514c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:39:53.704052 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506f84a6-d10d-4f33-8cf3-b8d9ba14514c admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:53.704330 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506f84a6-d10d-4f33-8cf3-b8d9ba14514c admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:53.704602 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506f84a6-d10d-4f33-8cf3-b8d9ba14514c admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:53.705221 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506f84a6-d10d-4f33-8cf3-b8d9ba14514c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:39:53.707121 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506f84a6-d10d-4f33-8cf3-b8d9ba14514c admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:39:53.726852 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1105/2216] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:39:53 2022] POST /identity/v3/projects => generated 377 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 31 12:39:53.732194 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1112/2217] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:39:53 2022] POST /identity/v3/users => generated 293 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 31 12:39:53.733257 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f72f61-0a42-4722-a90d-fa15c45bc848 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:39:53.733495 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506f84a6-d10d-4f33-8cf3-b8d9ba14514c admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:39:53.742904 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506f84a6-d10d-4f33-8cf3-b8d9ba14514c admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:39:53.760784 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:39:53.789935 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:39:53.800187 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f72f61-0a42-4722-a90d-fa15c45bc848 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:39:53.802772 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9b33fe-aa0d-440e-b89a-42e167ea9a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:39:53.803456 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9b33fe-aa0d-440e-b89a-42e167ea9a31 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:53.803739 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9b33fe-aa0d-440e-b89a-42e167ea9a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:53.804013 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9b33fe-aa0d-440e-b89a-42e167ea9a31 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:53.804676 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9b33fe-aa0d-440e-b89a-42e167ea9a31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:39:53.804931 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c00f1b-54ee-4baa-985a-5f8d53756a3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:39:53.805535 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c00f1b-54ee-4baa-985a-5f8d53756a3e admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:53.805812 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c00f1b-54ee-4baa-985a-5f8d53756a3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:53.806080 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c00f1b-54ee-4baa-985a-5f8d53756a3e admin admin] PATH_INFO: `/v3/users` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:53.806766 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9b33fe-aa0d-440e-b89a-42e167ea9a31 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:39:53.807478 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c00f1b-54ee-4baa-985a-5f8d53756a3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:39:53.809400 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c00f1b-54ee-4baa-985a-5f8d53756a3e admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:39:53.816741 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1113/2218] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:39:53 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 31 12:39:53.820762 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9b33fe-aa0d-440e-b89a-42e167ea9a31 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:39:53.831147 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9b33fe-aa0d-440e-b89a-42e167ea9a31 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:39:53.832608 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1106/2219] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:39:53 2022] POST /identity/v3/users => generated 295 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:39:53.836146 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50959b5a-ff0a-4977-9224-84738ec51bdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:39:53.836712 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c00f1b-54ee-4baa-985a-5f8d53756a3e admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:39:53.836846 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50959b5a-ff0a-4977-9224-84738ec51bdb admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:53.837211 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50959b5a-ff0a-4977-9224-84738ec51bdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:53.838070 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50959b5a-ff0a-4977-9224-84738ec51bdb admin admin] PATH_INFO: `/v3/projects/18862a8e83f54cc793c90928b7af39ce/users/44e56c28ef7a4b24baae60975d230d30/roles/176c07b4fe3b4537bdfeb4be21b379c5` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:53.846892 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c00f1b-54ee-4baa-985a-5f8d53756a3e admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:39:53.851583 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfc524f-e973-4755-bfc4-7fcd0387f9f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:39:53.852168 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfc524f-e973-4755-bfc4-7fcd0387f9f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:53.852446 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfc524f-e973-4755-bfc4-7fcd0387f9f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:53.852716 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfc524f-e973-4755-bfc4-7fcd0387f9f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:53.852891 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50959b5a-ff0a-4977-9224-84738ec51bdb admin admin] RBAC: Authorizing `identity:create_grant(project_id=18862a8e83f54cc793c90928b7af39ce, user_id=44e56c28ef7a4b24baae60975d230d30, role_id=176c07b4fe3b4537bdfeb4be21b379c5)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:39:53.853519 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfc524f-e973-4755-bfc4-7fcd0387f9f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:39:53.855289 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfc524f-e973-4755-bfc4-7fcd0387f9f0 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:39:53.855943 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50959b5a-ff0a-4977-9224-84738ec51bdb admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:39:53.866844 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1107/2220] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:39:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:39:53.870093 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1114/2221] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:39:53 2022] PUT /identity/v3/projects/18862a8e83f54cc793c90928b7af39ce/users/44e56c28ef7a4b24baae60975d230d30/roles/176c07b4fe3b4537bdfeb4be21b379c5 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 31 12:39:53.871769 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfc524f-e973-4755-bfc4-7fcd0387f9f0 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:39:53.873924 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50959b5a-ff0a-4977-9224-84738ec51bdb admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:39:53.881649 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfc524f-e973-4755-bfc4-7fcd0387f9f0 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:39:53.937166 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50959b5a-ff0a-4977-9224-84738ec51bdb admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:39:53.938437 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be350ee-c693-4696-907f-a8df8b0cde2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:39:53.939124 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be350ee-c693-4696-907f-a8df8b0cde2e admin admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:53.939564 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be350ee-c693-4696-907f-a8df8b0cde2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:53.939839 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be350ee-c693-4696-907f-a8df8b0cde2e admin admin] PATH_INFO: `/v3/projects/4ff613faa8f84a3480dae20f0eb6f3b2/users/955045275444497c88a3feb13cbd8e16/roles/176c07b4fe3b4537bdfeb4be21b379c5` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:53.942171 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25a8a73-f6cf-48cd-8e80-8a9bc7c51c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:39:53.942805 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25a8a73-f6cf-48cd-8e80-8a9bc7c51c2f admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:53.943086 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25a8a73-f6cf-48cd-8e80-8a9bc7c51c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:53.943437 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25a8a73-f6cf-48cd-8e80-8a9bc7c51c2f admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:53.944056 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25a8a73-f6cf-48cd-8e80-8a9bc7c51c2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:39:53.945715 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25a8a73-f6cf-48cd-8e80-8a9bc7c51c2f admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:39:53.955101 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be350ee-c693-4696-907f-a8df8b0cde2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ff613faa8f84a3480dae20f0eb6f3b2, user_id=955045275444497c88a3feb13cbd8e16, role_id=176c07b4fe3b4537bdfeb4be21b379c5)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:39:53.956484 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1115/2222] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:39:53 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 31 12:39:53.957954 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be350ee-c693-4696-907f-a8df8b0cde2e admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:39:53.960592 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25a8a73-f6cf-48cd-8e80-8a9bc7c51c2f admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:39:53.971704 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1108/2223] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:39:53 2022] PUT /identity/v3/projects/4ff613faa8f84a3480dae20f0eb6f3b2/users/955045275444497c88a3feb13cbd8e16/roles/176c07b4fe3b4537bdfeb4be21b379c5 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 31 12:39:53.972004 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25a8a73-f6cf-48cd-8e80-8a9bc7c51c2f admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:39:53.975547 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be350ee-c693-4696-907f-a8df8b0cde2e admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:39:54.029294 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29f77e8-3fc4-4cf8-ba53-f20a53fa301e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:39:54.029920 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29f77e8-3fc4-4cf8-ba53-f20a53fa301e admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:54.030218 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29f77e8-3fc4-4cf8-ba53-f20a53fa301e admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:54.030503 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29f77e8-3fc4-4cf8-ba53-f20a53fa301e admin admin] PATH_INFO: `/v3/projects/18862a8e83f54cc793c90928b7af39ce/users/44e56c28ef7a4b24baae60975d230d30/roles/176c07b4fe3b4537bdfeb4be21b379c5` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:54.032417 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29f77e8-3fc4-4cf8-ba53-f20a53fa301e admin admin] RBAC: Authorizing `identity:create_grant(project_id=18862a8e83f54cc793c90928b7af39ce, user_id=44e56c28ef7a4b24baae60975d230d30, role_id=176c07b4fe3b4537bdfeb4be21b379c5)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:39:54.035096 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29f77e8-3fc4-4cf8-ba53-f20a53fa301e admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:39:54.039068 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be350ee-c693-4696-907f-a8df8b0cde2e admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:39:54.045956 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1116/2224] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:39:53 2022] PUT /identity/v3/projects/18862a8e83f54cc793c90928b7af39ce/users/44e56c28ef7a4b24baae60975d230d30/roles/176c07b4fe3b4537bdfeb4be21b379c5 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 31 12:39:54.056171 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315ce6d2-26e9-4f4d-9c36-5fed11d5ecff None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:54.056611 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315ce6d2-26e9-4f4d-9c36-5fed11d5ecff None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:54.056895 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315ce6d2-26e9-4f4d-9c36-5fed11d5ecff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:54.074454 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:39:54.084870 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-315ce6d2-26e9-4f4d-9c36-5fed11d5ecff None None] MFA Rules not processed for user `44e56c28ef7a4b24baae60975d230d30`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:39:54.086087 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-315ce6d2-26e9-4f4d-9c36-5fed11d5ecff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:39:54.104945 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4a9856-13d0-40ff-9d2c-a45383d906af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:39:54.105553 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4a9856-13d0-40ff-9d2c-a45383d906af admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:54.105836 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4a9856-13d0-40ff-9d2c-a45383d906af admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:54.106103 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4a9856-13d0-40ff-9d2c-a45383d906af admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:54.106734 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4a9856-13d0-40ff-9d2c-a45383d906af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:39:54.108331 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4a9856-13d0-40ff-9d2c-a45383d906af admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:39:54.118233 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1109/2225] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:39:53 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 31 12:39:54.123775 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4a9856-13d0-40ff-9d2c-a45383d906af admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:39:54.129497 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:39:54.135395 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4a9856-13d0-40ff-9d2c-a45383d906af admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:39:54.140726 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250841a4-a970-4214-9462-fa065d90784b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:39:54.141547 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250841a4-a970-4214-9462-fa065d90784b admin admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:54.142135 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250841a4-a970-4214-9462-fa065d90784b admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:54.142425 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250841a4-a970-4214-9462-fa065d90784b admin admin] PATH_INFO: `/v3/projects/4ff613faa8f84a3480dae20f0eb6f3b2/users/955045275444497c88a3feb13cbd8e16/roles/176c07b4fe3b4537bdfeb4be21b379c5` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:54.144655 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250841a4-a970-4214-9462-fa065d90784b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ff613faa8f84a3480dae20f0eb6f3b2, user_id=955045275444497c88a3feb13cbd8e16, role_id=176c07b4fe3b4537bdfeb4be21b379c5)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:39:54.147346 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250841a4-a970-4214-9462-fa065d90784b admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:39:54.158757 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1110/2226] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:39:54 2022] PUT /identity/v3/projects/4ff613faa8f84a3480dae20f0eb6f3b2/users/955045275444497c88a3feb13cbd8e16/roles/176c07b4fe3b4537bdfeb4be21b379c5 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 31 12:39:54.159968 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1117/2227] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:39:54 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 31 12:39:54.169965 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7af2174-2abc-4f4d-b783-d5706b0544f0 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:54.170256 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7af2174-2abc-4f4d-b783-d5706b0544f0 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:54.170531 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7af2174-2abc-4f4d-b783-d5706b0544f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:54.174562 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315ce6d2-26e9-4f4d-9c36-5fed11d5ecff None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:39:54.176078 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-315ce6d2-26e9-4f4d-9c36-5fed11d5ecff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:39:54.199472 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-e7af2174-2abc-4f4d-b783-d5706b0544f0 None None] MFA Rules not processed for user `955045275444497c88a3feb13cbd8e16`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:39:54.200580 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-e7af2174-2abc-4f4d-b783-d5706b0544f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:39:54.234633 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315ce6d2-26e9-4f4d-9c36-5fed11d5ecff None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:39:54.273222 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1111/2228] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:39:54 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:39:54.285559 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9071b95a-10c0-4df0-b6eb-8446d64adeeb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d6d7ea077d49e8a66c6c6762499702', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:39:54.285722 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7af2174-2abc-4f4d-b783-d5706b0544f0 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:39:54.286178 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9071b95a-10c0-4df0-b6eb-8446d64adeeb service cinder] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:54.286457 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9071b95a-10c0-4df0-b6eb-8446d64adeeb service cinder] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:54.286730 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9071b95a-10c0-4df0-b6eb-8446d64adeeb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:54.287080 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-e7af2174-2abc-4f4d-b783-d5706b0544f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:39:54.299497 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7af2174-2abc-4f4d-b783-d5706b0544f0 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:39:54.303097 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a50505-39aa-4e94-afa9-7a3292da8e48 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d6d7ea077d49e8a66c6c6762499702', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:39:54.303732 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a50505-39aa-4e94-afa9-7a3292da8e48 service cinder] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:54.304012 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a50505-39aa-4e94-afa9-7a3292da8e48 service cinder] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:54.304280 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a50505-39aa-4e94-afa9-7a3292da8e48 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:54.314323 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a50505-39aa-4e94-afa9-7a3292da8e48 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:39:54.315770 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a50505-39aa-4e94-afa9-7a3292da8e48 service cinder] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:39:54.326083 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1112/2229] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:39:54 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 31 12:39:54.351535 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9071b95a-10c0-4df0-b6eb-8446d64adeeb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:39:54.352959 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9071b95a-10c0-4df0-b6eb-8446d64adeeb service cinder] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:39:54.365025 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1118/2230] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:39:54 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 31 12:39:54.403420 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a50505-39aa-4e94-afa9-7a3292da8e48 service cinder] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:39:54.412050 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a50505-39aa-4e94-afa9-7a3292da8e48 service cinder] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:39:54.416555 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f2e3d3-8425-43f2-bc1a-4d5bd62729bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d6d7ea077d49e8a66c6c6762499702', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:39:54.417163 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f2e3d3-8425-43f2-bc1a-4d5bd62729bc service cinder] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:54.417445 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f2e3d3-8425-43f2-bc1a-4d5bd62729bc service cinder] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:54.417720 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f2e3d3-8425-43f2-bc1a-4d5bd62729bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:54.445832 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9071b95a-10c0-4df0-b6eb-8446d64adeeb service cinder] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:39:54.454919 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9071b95a-10c0-4df0-b6eb-8446d64adeeb service cinder] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:39:54.458575 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4286cb5-d4e4-4c90-84a4-82cdfae8e660 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d6d7ea077d49e8a66c6c6762499702', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:39:54.459218 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4286cb5-d4e4-4c90-84a4-82cdfae8e660 service cinder] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:39:54.459483 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4286cb5-d4e4-4c90-84a4-82cdfae8e660 service cinder] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:39:54.459754 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4286cb5-d4e4-4c90-84a4-82cdfae8e660 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:39:54.484422 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f2e3d3-8425-43f2-bc1a-4d5bd62729bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:39:54.485845 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f2e3d3-8425-43f2-bc1a-4d5bd62729bc service cinder] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:39:54.495871 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1113/2231] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:39:54 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:39:54.501497 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:39:54.519553 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4286cb5-d4e4-4c90-84a4-82cdfae8e660 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:39:54.520968 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4286cb5-d4e4-4c90-84a4-82cdfae8e660 service cinder] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:39:54.531500 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1119/2232] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:39:54 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:40:04.490758 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=88,keystone:INSERT=1 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:40:04.526687 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=9 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:40:05.465546 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f2e3d3-8425-43f2-bc1a-4d5bd62729bc service cinder] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:05.534064 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f2e3d3-8425-43f2-bc1a-4d5bd62729bc service cinder] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:05.553304 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea99cd3-72b9-4057-b846-db8cad26036a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:05.553304 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea99cd3-72b9-4057-b846-db8cad26036a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:05.553304 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea99cd3-72b9-4057-b846-db8cad26036a admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:05.553304 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea99cd3-72b9-4057-b846-db8cad26036a admin admin] PATH_INFO: `/v3/users/16274992f3ec49cfbf2e939ebdaf5241` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:05.553675 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea99cd3-72b9-4057-b846-db8cad26036a admin admin] RBAC: Authorizing `identity:delete_user(user_id=16274992f3ec49cfbf2e939ebdaf5241)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:05.553675 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea99cd3-72b9-4057-b846-db8cad26036a admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:05.593369 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-1ea99cd3-72b9-4057-b846-db8cad26036a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16274992f3ec49cfbf2e939ebdaf5241'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:05.600544 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-1ea99cd3-72b9-4057-b846-db8cad26036a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16274992f3ec49cfbf2e939ebdaf5241'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:05.601201 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-1ea99cd3-72b9-4057-b846-db8cad26036a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16274992f3ec49cfbf2e939ebdaf5241'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:05.613679 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-1ea99cd3-72b9-4057-b846-db8cad26036a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16274992f3ec49cfbf2e939ebdaf5241'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:05.621603 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-1ea99cd3-72b9-4057-b846-db8cad26036a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16274992f3ec49cfbf2e939ebdaf5241'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:05.627849 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1114/2233] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:40:05 2022] DELETE /identity/v3/users/16274992f3ec49cfbf2e939ebdaf5241 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:40:05.632284 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51580d0d-73a1-4570-90fd-14cfa40f6c9c None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:05.632773 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51580d0d-73a1-4570-90fd-14cfa40f6c9c None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:05.632869 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51580d0d-73a1-4570-90fd-14cfa40f6c9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:05.653335 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-51580d0d-73a1-4570-90fd-14cfa40f6c9c None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:40:05.654559 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-51580d0d-73a1-4570-90fd-14cfa40f6c9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:05.714379 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1120/2234] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:40:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:40:05.723879 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea99cd3-72b9-4057-b846-db8cad26036a admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:05.725255 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-1ea99cd3-72b9-4057-b846-db8cad26036a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:05.783139 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea99cd3-72b9-4057-b846-db8cad26036a admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:05.787199 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eb373c-5e60-4f66-b043-a28e5e4fef98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:05.787694 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eb373c-5e60-4f66-b043-a28e5e4fef98 service neutron] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:05.787994 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eb373c-5e60-4f66-b043-a28e5e4fef98 service neutron] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:05.788239 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eb373c-5e60-4f66-b043-a28e5e4fef98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:05.798215 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eb373c-5e60-4f66-b043-a28e5e4fef98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:05.799682 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eb373c-5e60-4f66-b043-a28e5e4fef98 service neutron] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:05.809230 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1115/2235] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:40:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:40:06.080154 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51580d0d-73a1-4570-90fd-14cfa40f6c9c None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:06.081590 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-51580d0d-73a1-4570-90fd-14cfa40f6c9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:06.093504 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51580d0d-73a1-4570-90fd-14cfa40f6c9c None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:06.097975 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f691426c-afde-44f9-a917-4fcdcdcc10c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:06.099385 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f691426c-afde-44f9-a917-4fcdcdcc10c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:06.099671 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f691426c-afde-44f9-a917-4fcdcdcc10c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:06.100114 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f691426c-afde-44f9-a917-4fcdcdcc10c4 admin admin] PATH_INFO: `/v3/projects/18ce8d2885af4573ae11a903c4be8a1f` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:06.101317 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f691426c-afde-44f9-a917-4fcdcdcc10c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=18ce8d2885af4573ae11a903c4be8a1f)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:06.104336 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f691426c-afde-44f9-a917-4fcdcdcc10c4 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:06.152541 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-f691426c-afde-44f9-a917-4fcdcdcc10c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18ce8d2885af4573ae11a903c4be8a1f', 'request_id': 'req-f691426c-afde-44f9-a917-4fcdcdcc10c4'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:06.158618 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-f691426c-afde-44f9-a917-4fcdcdcc10c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18ce8d2885af4573ae11a903c4be8a1f', 'request_id': 'req-f691426c-afde-44f9-a917-4fcdcdcc10c4'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:06.166614 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-f691426c-afde-44f9-a917-4fcdcdcc10c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18ce8d2885af4573ae11a903c4be8a1f', 'request_id': 'req-f691426c-afde-44f9-a917-4fcdcdcc10c4'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:06.176885 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-f691426c-afde-44f9-a917-4fcdcdcc10c4 admin admin] The token cache is being invalidate because project 18ce8d2885af4573ae11a903c4be8a1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:40:06.177173 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-f691426c-afde-44f9-a917-4fcdcdcc10c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:06.177789 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-f691426c-afde-44f9-a917-4fcdcdcc10c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:06.179288 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1121/2236] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:40:06 2022] DELETE /identity/v3/projects/18ce8d2885af4573ae11a903c4be8a1f => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:40:06.183951 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eb373c-5e60-4f66-b043-a28e5e4fef98 service neutron] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:06.193463 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eb373c-5e60-4f66-b043-a28e5e4fef98 service neutron] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:06.211129 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:40:06.250140 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc25f0e6-cac4-42d3-9bbe-546c0a041ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:06.250768 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc25f0e6-cac4-42d3-9bbe-546c0a041ed6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:06.251047 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc25f0e6-cac4-42d3-9bbe-546c0a041ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:06.251360 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc25f0e6-cac4-42d3-9bbe-546c0a041ed6 admin admin] PATH_INFO: `/v3/users/955045275444497c88a3feb13cbd8e16` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:06.252470 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc25f0e6-cac4-42d3-9bbe-546c0a041ed6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=955045275444497c88a3feb13cbd8e16)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:06.254367 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc25f0e6-cac4-42d3-9bbe-546c0a041ed6 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:06.300467 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-cc25f0e6-cac4-42d3-9bbe-546c0a041ed6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '955045275444497c88a3feb13cbd8e16'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:06.308150 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-cc25f0e6-cac4-42d3-9bbe-546c0a041ed6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '955045275444497c88a3feb13cbd8e16'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:06.308777 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-cc25f0e6-cac4-42d3-9bbe-546c0a041ed6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '955045275444497c88a3feb13cbd8e16'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:06.320644 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-cc25f0e6-cac4-42d3-9bbe-546c0a041ed6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '955045275444497c88a3feb13cbd8e16'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:06.327634 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-cc25f0e6-cac4-42d3-9bbe-546c0a041ed6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '955045275444497c88a3feb13cbd8e16'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:06.333279 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1116/2237] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:40:06 2022] DELETE /identity/v3/users/955045275444497c88a3feb13cbd8e16 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:40:06.606376 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f691426c-afde-44f9-a917-4fcdcdcc10c4 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:06.607844 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-f691426c-afde-44f9-a917-4fcdcdcc10c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:06.621099 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f691426c-afde-44f9-a917-4fcdcdcc10c4 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:06.678341 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a91928-7d29-4472-9460-cdbd1941711b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:06.678959 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a91928-7d29-4472-9460-cdbd1941711b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:06.679271 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a91928-7d29-4472-9460-cdbd1941711b admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:06.679548 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a91928-7d29-4472-9460-cdbd1941711b admin admin] PATH_INFO: `/v3/projects/4ff613faa8f84a3480dae20f0eb6f3b2` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:06.680649 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a91928-7d29-4472-9460-cdbd1941711b admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ff613faa8f84a3480dae20f0eb6f3b2)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:06.683133 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a91928-7d29-4472-9460-cdbd1941711b admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:06.712142 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:40:06.741069 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-a2a91928-7d29-4472-9460-cdbd1941711b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ff613faa8f84a3480dae20f0eb6f3b2', 'request_id': 'req-a2a91928-7d29-4472-9460-cdbd1941711b'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:06.747940 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-a2a91928-7d29-4472-9460-cdbd1941711b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ff613faa8f84a3480dae20f0eb6f3b2', 'request_id': 'req-a2a91928-7d29-4472-9460-cdbd1941711b'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:06.756137 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-a2a91928-7d29-4472-9460-cdbd1941711b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ff613faa8f84a3480dae20f0eb6f3b2', 'request_id': 'req-a2a91928-7d29-4472-9460-cdbd1941711b'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:06.767395 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-a2a91928-7d29-4472-9460-cdbd1941711b admin admin] The token cache is being invalidate because project 4ff613faa8f84a3480dae20f0eb6f3b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:40:06.767690 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-a2a91928-7d29-4472-9460-cdbd1941711b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:06.768576 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-a2a91928-7d29-4472-9460-cdbd1941711b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:06.770159 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1122/2238] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:40:06 2022] DELETE /identity/v3/projects/4ff613faa8f84a3480dae20f0eb6f3b2 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:40:06.775133 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b70b873-aa58-4b7e-84f7-3b88f7996584 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:06.775703 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b70b873-aa58-4b7e-84f7-3b88f7996584 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:06.775950 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b70b873-aa58-4b7e-84f7-3b88f7996584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:06.797894 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-1b70b873-aa58-4b7e-84f7-3b88f7996584 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:40:06.799486 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-1b70b873-aa58-4b7e-84f7-3b88f7996584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:06.863098 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1117/2239] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:40:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:40:06.871687 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68ca839-f408-432d-979a-83cb85384f8c None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:06.871969 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68ca839-f408-432d-979a-83cb85384f8c None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:06.872293 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68ca839-f408-432d-979a-83cb85384f8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:06.892802 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-d68ca839-f408-432d-979a-83cb85384f8c None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:40:06.894252 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-d68ca839-f408-432d-979a-83cb85384f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:06.903091 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1123/2240] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:40:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:40:06.907499 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b70b873-aa58-4b7e-84f7-3b88f7996584 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:06.918009 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b70b873-aa58-4b7e-84f7-3b88f7996584 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:06.923315 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d1dc48-28b7-4412-b6d6-f10d9e9f01d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:06.924107 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d1dc48-28b7-4412-b6d6-f10d9e9f01d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:06.924454 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d1dc48-28b7-4412-b6d6-f10d9e9f01d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:06.924796 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d1dc48-28b7-4412-b6d6-f10d9e9f01d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:06.925548 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d1dc48-28b7-4412-b6d6-f10d9e9f01d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:40:06.925914 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d1dc48-28b7-4412-b6d6-f10d9e9f01d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:06.927612 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d1dc48-28b7-4412-b6d6-f10d9e9f01d1 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:06.940160 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1118/2241] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Oct 31 12:40:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:40:06.945003 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68ca839-f408-432d-979a-83cb85384f8c None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:06.955952 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68ca839-f408-432d-979a-83cb85384f8c None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:06.960897 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5cd3f2-04de-4029-899f-9674c258e70e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:06.961496 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5cd3f2-04de-4029-899f-9674c258e70e admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:06.961769 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5cd3f2-04de-4029-899f-9674c258e70e admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:06.962038 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5cd3f2-04de-4029-899f-9674c258e70e admin admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:06.963302 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5cd3f2-04de-4029-899f-9674c258e70e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:06.965577 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5cd3f2-04de-4029-899f-9674c258e70e admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:06.993450 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1124/2242] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:40:06 2022] POST /identity/v3/projects => generated 401 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:40:06.998473 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d1dc48-28b7-4412-b6d6-f10d9e9f01d1 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:07.008939 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d1dc48-28b7-4412-b6d6-f10d9e9f01d1 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:07.036091 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:40:07.068301 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1098467-373c-42fc-8083-f5caae8d4d8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:07.068842 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1098467-373c-42fc-8083-f5caae8d4d8a admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:07.069127 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1098467-373c-42fc-8083-f5caae8d4d8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:07.069407 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1098467-373c-42fc-8083-f5caae8d4d8a admin admin] PATH_INFO: `/v3/users` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:07.070040 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1098467-373c-42fc-8083-f5caae8d4d8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:07.072602 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1098467-373c-42fc-8083-f5caae8d4d8a admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:07.096517 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1119/2243] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:40:06 2022] POST /identity/v3/users => generated 301 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:40:07.097744 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5cd3f2-04de-4029-899f-9674c258e70e admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:07.107843 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5cd3f2-04de-4029-899f-9674c258e70e admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:07.112332 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86073fec-61fc-4373-8387-3a49fd0bc9ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:07.112920 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86073fec-61fc-4373-8387-3a49fd0bc9ca admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:07.113240 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86073fec-61fc-4373-8387-3a49fd0bc9ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:07.113457 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86073fec-61fc-4373-8387-3a49fd0bc9ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:07.114084 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86073fec-61fc-4373-8387-3a49fd0bc9ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:07.115718 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86073fec-61fc-4373-8387-3a49fd0bc9ca admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:07.125855 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1125/2244] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:40:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 31 12:40:07.129736 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1098467-373c-42fc-8083-f5caae8d4d8a admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:07.140070 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1098467-373c-42fc-8083-f5caae8d4d8a admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:07.144878 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3db26a-cb82-48f2-8911-8432d09a195a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:07.145459 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3db26a-cb82-48f2-8911-8432d09a195a admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:07.145734 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3db26a-cb82-48f2-8911-8432d09a195a admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:07.146000 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3db26a-cb82-48f2-8911-8432d09a195a admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:07.146630 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3db26a-cb82-48f2-8911-8432d09a195a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:07.148234 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3db26a-cb82-48f2-8911-8432d09a195a admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:07.158677 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1120/2245] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:40:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 31 12:40:07.162615 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86073fec-61fc-4373-8387-3a49fd0bc9ca admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:07.173853 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86073fec-61fc-4373-8387-3a49fd0bc9ca admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:07.178404 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596aae64-2f55-4d86-bec7-c99964aa56a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:07.179045 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596aae64-2f55-4d86-bec7-c99964aa56a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:07.179360 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596aae64-2f55-4d86-bec7-c99964aa56a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:07.179634 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596aae64-2f55-4d86-bec7-c99964aa56a5 admin admin] PATH_INFO: `/v3/projects/d332a5b363914895ab2f3e68ed25220c/users/a042399cf9454b8a871cfc5dae9f290a/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:07.194417 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596aae64-2f55-4d86-bec7-c99964aa56a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d332a5b363914895ab2f3e68ed25220c, user_id=a042399cf9454b8a871cfc5dae9f290a, role_id=ca1a5695f8bd499aa0b8cedaeecd2cc6)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:07.197741 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596aae64-2f55-4d86-bec7-c99964aa56a5 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:07.210239 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1126/2246] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:40:07 2022] PUT /identity/v3/projects/d332a5b363914895ab2f3e68ed25220c/users/a042399cf9454b8a871cfc5dae9f290a/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:40:07.221415 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead8481a-b05f-429c-9ae8-aca1506f3180 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:07.221695 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead8481a-b05f-429c-9ae8-aca1506f3180 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:07.221965 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead8481a-b05f-429c-9ae8-aca1506f3180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:07.249211 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-ead8481a-b05f-429c-9ae8-aca1506f3180 None None] MFA Rules not processed for user `a042399cf9454b8a871cfc5dae9f290a`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:40:07.250426 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-ead8481a-b05f-429c-9ae8-aca1506f3180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:07.316796 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1121/2247] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:40:07 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:40:07.321228 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596aae64-2f55-4d86-bec7-c99964aa56a5 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:07.383231 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596aae64-2f55-4d86-bec7-c99964aa56a5 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:07.388366 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68f45f4-353c-42b0-a5dd-195121bfe958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:07.388983 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68f45f4-353c-42b0-a5dd-195121bfe958 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:07.389245 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68f45f4-353c-42b0-a5dd-195121bfe958 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:07.389516 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68f45f4-353c-42b0-a5dd-195121bfe958 admin admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:07.390146 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68f45f4-353c-42b0-a5dd-195121bfe958 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:07.392094 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68f45f4-353c-42b0-a5dd-195121bfe958 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:07.422046 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1127/2248] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:40:07 2022] POST /identity/v3/projects => generated 399 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:40:07.426142 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead8481a-b05f-429c-9ae8-aca1506f3180 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:07.436812 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead8481a-b05f-429c-9ae8-aca1506f3180 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:07.493417 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c675bf9-9be2-4df1-ba16-9481dedba007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:07.494483 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c675bf9-9be2-4df1-ba16-9481dedba007 admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:07.494869 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c675bf9-9be2-4df1-ba16-9481dedba007 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:07.494978 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c675bf9-9be2-4df1-ba16-9481dedba007 admin admin] PATH_INFO: `/v3/users` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:07.495640 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c675bf9-9be2-4df1-ba16-9481dedba007 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:07.497587 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c675bf9-9be2-4df1-ba16-9481dedba007 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:07.522745 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1122/2249] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:40:07 2022] POST /identity/v3/users => generated 306 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:40:07.524934 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68f45f4-353c-42b0-a5dd-195121bfe958 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:07.535998 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68f45f4-353c-42b0-a5dd-195121bfe958 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:07.542948 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ad7c34-20c2-49b9-b391-edb46909cc77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:07.543779 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ad7c34-20c2-49b9-b391-edb46909cc77 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:07.544066 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ad7c34-20c2-49b9-b391-edb46909cc77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:07.544339 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ad7c34-20c2-49b9-b391-edb46909cc77 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:07.544974 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ad7c34-20c2-49b9-b391-edb46909cc77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:07.546556 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ad7c34-20c2-49b9-b391-edb46909cc77 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:07.559862 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1128/2250] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:40:07 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:40:07.563971 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c675bf9-9be2-4df1-ba16-9481dedba007 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:07.576022 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c675bf9-9be2-4df1-ba16-9481dedba007 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:07.580802 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29533c8-c795-411d-8381-8be9c4e473d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:07.581456 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29533c8-c795-411d-8381-8be9c4e473d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:07.581732 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29533c8-c795-411d-8381-8be9c4e473d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:07.582016 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29533c8-c795-411d-8381-8be9c4e473d6 admin admin] PATH_INFO: `/v3/projects/50af65287f9d4a398c1cf93fe51f7b1c/users/79bfd1d60cc34bdc872566340e8d3a7a/roles/176c07b4fe3b4537bdfeb4be21b379c5` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:07.598369 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29533c8-c795-411d-8381-8be9c4e473d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50af65287f9d4a398c1cf93fe51f7b1c, user_id=79bfd1d60cc34bdc872566340e8d3a7a, role_id=176c07b4fe3b4537bdfeb4be21b379c5)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:07.601092 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29533c8-c795-411d-8381-8be9c4e473d6 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:07.615381 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1123/2251] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:40:07 2022] PUT /identity/v3/projects/50af65287f9d4a398c1cf93fe51f7b1c/users/79bfd1d60cc34bdc872566340e8d3a7a/roles/176c07b4fe3b4537bdfeb4be21b379c5 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:40:07.619814 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ad7c34-20c2-49b9-b391-edb46909cc77 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:07.629856 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ad7c34-20c2-49b9-b391-edb46909cc77 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:07.651591 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:40:07.694281 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2084b6d-de87-46a4-8cdd-e82631516f08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:07.694992 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2084b6d-de87-46a4-8cdd-e82631516f08 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:07.695144 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2084b6d-de87-46a4-8cdd-e82631516f08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:07.695438 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2084b6d-de87-46a4-8cdd-e82631516f08 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:07.696372 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2084b6d-de87-46a4-8cdd-e82631516f08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:07.698946 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2084b6d-de87-46a4-8cdd-e82631516f08 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:07.709955 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1129/2252] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:40:07 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 31 12:40:07.714877 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29533c8-c795-411d-8381-8be9c4e473d6 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:07.720564 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:40:07.727466 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29533c8-c795-411d-8381-8be9c4e473d6 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:07.732639 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2367ec44-d23f-4b4d-b516-d891e89b137b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:07.733290 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2367ec44-d23f-4b4d-b516-d891e89b137b admin admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:07.733604 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2367ec44-d23f-4b4d-b516-d891e89b137b admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:07.733871 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2367ec44-d23f-4b4d-b516-d891e89b137b admin admin] PATH_INFO: `/v3/projects/50af65287f9d4a398c1cf93fe51f7b1c/users/79bfd1d60cc34bdc872566340e8d3a7a/roles/176c07b4fe3b4537bdfeb4be21b379c5` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:07.735925 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2367ec44-d23f-4b4d-b516-d891e89b137b admin admin] RBAC: Authorizing `identity:create_grant(project_id=50af65287f9d4a398c1cf93fe51f7b1c, user_id=79bfd1d60cc34bdc872566340e8d3a7a, role_id=176c07b4fe3b4537bdfeb4be21b379c5)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:07.738543 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2367ec44-d23f-4b4d-b516-d891e89b137b admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:07.751390 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1124/2253] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:40:07 2022] PUT /identity/v3/projects/50af65287f9d4a398c1cf93fe51f7b1c/users/79bfd1d60cc34bdc872566340e8d3a7a/roles/176c07b4fe3b4537bdfeb4be21b379c5 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:40:07.763050 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ae00d0-b7e6-4e5d-b863-4b012f3663a1 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:07.763412 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ae00d0-b7e6-4e5d-b863-4b012f3663a1 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:07.763796 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ae00d0-b7e6-4e5d-b863-4b012f3663a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:07.798171 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-09ae00d0-b7e6-4e5d-b863-4b012f3663a1 None None] MFA Rules not processed for user `79bfd1d60cc34bdc872566340e8d3a7a`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:40:07.800209 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-09ae00d0-b7e6-4e5d-b863-4b012f3663a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:07.878561 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1130/2254] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:40:07 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:40:07.890250 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2367ec44-d23f-4b4d-b516-d891e89b137b admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:07.892286 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-2367ec44-d23f-4b4d-b516-d891e89b137b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:07.969093 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2367ec44-d23f-4b4d-b516-d891e89b137b admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:07.973121 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9485dbf2-4f54-491e-a0c1-dff590ea1ffa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d6d7ea077d49e8a66c6c6762499702', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:07.973732 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9485dbf2-4f54-491e-a0c1-dff590ea1ffa service cinder] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:07.974015 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9485dbf2-4f54-491e-a0c1-dff590ea1ffa service cinder] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:07.974284 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9485dbf2-4f54-491e-a0c1-dff590ea1ffa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:07.983662 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9485dbf2-4f54-491e-a0c1-dff590ea1ffa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:07.985093 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9485dbf2-4f54-491e-a0c1-dff590ea1ffa service cinder] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:07.997018 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1125/2255] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:40:07 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:40:08.097747 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ae00d0-b7e6-4e5d-b863-4b012f3663a1 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:08.099354 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-09ae00d0-b7e6-4e5d-b863-4b012f3663a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:08.112556 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ae00d0-b7e6-4e5d-b863-4b012f3663a1 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:08.116229 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c433d4db-c2e9-4628-b788-dc5582faaced service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d6d7ea077d49e8a66c6c6762499702', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:08.116897 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c433d4db-c2e9-4628-b788-dc5582faaced service cinder] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:08.117126 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c433d4db-c2e9-4628-b788-dc5582faaced service cinder] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:08.117394 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c433d4db-c2e9-4628-b788-dc5582faaced service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:08.189630 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c433d4db-c2e9-4628-b788-dc5582faaced service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:08.191739 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c433d4db-c2e9-4628-b788-dc5582faaced service cinder] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:08.202585 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1131/2256] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:40:08 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:40:12.346008 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9485dbf2-4f54-491e-a0c1-dff590ea1ffa service cinder] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:12.422231 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9485dbf2-4f54-491e-a0c1-dff590ea1ffa service cinder] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:12.427930 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb1f71b-d848-4743-86e3-c451ba73083a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:12.428751 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb1f71b-d848-4743-86e3-c451ba73083a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:12.429174 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb1f71b-d848-4743-86e3-c451ba73083a admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:12.429594 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb1f71b-d848-4743-86e3-c451ba73083a admin admin] PATH_INFO: `/v3/users/a042399cf9454b8a871cfc5dae9f290a` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:12.431999 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb1f71b-d848-4743-86e3-c451ba73083a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a042399cf9454b8a871cfc5dae9f290a)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:12.434434 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb1f71b-d848-4743-86e3-c451ba73083a admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:12.505031 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-fcb1f71b-d848-4743-86e3-c451ba73083a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a042399cf9454b8a871cfc5dae9f290a'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:12.513126 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-fcb1f71b-d848-4743-86e3-c451ba73083a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a042399cf9454b8a871cfc5dae9f290a'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:12.513887 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-fcb1f71b-d848-4743-86e3-c451ba73083a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a042399cf9454b8a871cfc5dae9f290a'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:12.526953 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-fcb1f71b-d848-4743-86e3-c451ba73083a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a042399cf9454b8a871cfc5dae9f290a'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:12.533978 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-fcb1f71b-d848-4743-86e3-c451ba73083a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a042399cf9454b8a871cfc5dae9f290a'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:12.540878 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1126/2257] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:40:12 2022] DELETE /identity/v3/users/a042399cf9454b8a871cfc5dae9f290a => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 31 12:40:12.545025 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9013df-ba0e-4aa0-8657-500a70a5e58d None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:12.545505 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9013df-ba0e-4aa0-8657-500a70a5e58d None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:12.545505 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9013df-ba0e-4aa0-8657-500a70a5e58d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:12.565695 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-4a9013df-ba0e-4aa0-8657-500a70a5e58d None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:40:12.566926 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-4a9013df-ba0e-4aa0-8657-500a70a5e58d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:12.607309 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:40:12.628031 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1132/2258] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:40:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:40:12.638244 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb1f71b-d848-4743-86e3-c451ba73083a admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:12.639670 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-fcb1f71b-d848-4743-86e3-c451ba73083a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:12.653400 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:40:12.705283 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb1f71b-d848-4743-86e3-c451ba73083a admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:12.709373 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b0a8a0-01e4-490b-aa75-9bb53a081ca9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:12.709982 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b0a8a0-01e4-490b-aa75-9bb53a081ca9 service neutron] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:12.710263 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b0a8a0-01e4-490b-aa75-9bb53a081ca9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:12.710535 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b0a8a0-01e4-490b-aa75-9bb53a081ca9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:12.719991 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b0a8a0-01e4-490b-aa75-9bb53a081ca9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:12.721397 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b0a8a0-01e4-490b-aa75-9bb53a081ca9 service neutron] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:12.731437 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1127/2259] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:40:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 31 12:40:13.178906 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9013df-ba0e-4aa0-8657-500a70a5e58d None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:13.183585 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-4a9013df-ba0e-4aa0-8657-500a70a5e58d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:13.224367 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9013df-ba0e-4aa0-8657-500a70a5e58d None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:13.233805 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2230f992-7d61-4665-b045-7278870d1d1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:13.234443 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2230f992-7d61-4665-b045-7278870d1d1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:13.234720 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2230f992-7d61-4665-b045-7278870d1d1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:13.235003 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2230f992-7d61-4665-b045-7278870d1d1c admin admin] PATH_INFO: `/v3/projects/d332a5b363914895ab2f3e68ed25220c` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:13.236744 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2230f992-7d61-4665-b045-7278870d1d1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d332a5b363914895ab2f3e68ed25220c)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:13.239033 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2230f992-7d61-4665-b045-7278870d1d1c admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:13.322251 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-2230f992-7d61-4665-b045-7278870d1d1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd332a5b363914895ab2f3e68ed25220c', 'request_id': 'req-2230f992-7d61-4665-b045-7278870d1d1c'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:13.329824 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-2230f992-7d61-4665-b045-7278870d1d1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd332a5b363914895ab2f3e68ed25220c', 'request_id': 'req-2230f992-7d61-4665-b045-7278870d1d1c'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:13.341432 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-2230f992-7d61-4665-b045-7278870d1d1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd332a5b363914895ab2f3e68ed25220c', 'request_id': 'req-2230f992-7d61-4665-b045-7278870d1d1c'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:13.353513 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-2230f992-7d61-4665-b045-7278870d1d1c admin admin] The token cache is being invalidate because project d332a5b363914895ab2f3e68ed25220c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:40:13.354337 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-2230f992-7d61-4665-b045-7278870d1d1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:13.354612 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-2230f992-7d61-4665-b045-7278870d1d1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:13.356388 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1133/2260] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:40:13 2022] DELETE /identity/v3/projects/d332a5b363914895ab2f3e68ed25220c => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:40:13.360332 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b0a8a0-01e4-490b-aa75-9bb53a081ca9 service neutron] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:13.371958 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b0a8a0-01e4-490b-aa75-9bb53a081ca9 service neutron] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:13.434555 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1122686-6fa9-473b-9c53-93a5d33f1d1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:13.435202 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1122686-6fa9-473b-9c53-93a5d33f1d1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:13.435483 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1122686-6fa9-473b-9c53-93a5d33f1d1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:13.435752 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1122686-6fa9-473b-9c53-93a5d33f1d1d admin admin] PATH_INFO: `/v3/users/79bfd1d60cc34bdc872566340e8d3a7a` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:13.436876 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1122686-6fa9-473b-9c53-93a5d33f1d1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=79bfd1d60cc34bdc872566340e8d3a7a)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:13.438859 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1122686-6fa9-473b-9c53-93a5d33f1d1d admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:13.490550 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-f1122686-6fa9-473b-9c53-93a5d33f1d1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79bfd1d60cc34bdc872566340e8d3a7a'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:13.502054 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-f1122686-6fa9-473b-9c53-93a5d33f1d1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79bfd1d60cc34bdc872566340e8d3a7a'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:13.502820 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-f1122686-6fa9-473b-9c53-93a5d33f1d1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79bfd1d60cc34bdc872566340e8d3a7a'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:13.522274 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-f1122686-6fa9-473b-9c53-93a5d33f1d1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79bfd1d60cc34bdc872566340e8d3a7a'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:13.531276 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-f1122686-6fa9-473b-9c53-93a5d33f1d1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79bfd1d60cc34bdc872566340e8d3a7a'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:13.543612 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1128/2261] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:40:13 2022] DELETE /identity/v3/users/79bfd1d60cc34bdc872566340e8d3a7a => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:40:13.854658 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2230f992-7d61-4665-b045-7278870d1d1c admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:13.857066 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-2230f992-7d61-4665-b045-7278870d1d1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:13.874369 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2230f992-7d61-4665-b045-7278870d1d1c admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:13.940781 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64de148b-68ea-4c76-b4e6-bce15aafd5d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:13.941431 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64de148b-68ea-4c76-b4e6-bce15aafd5d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:13.941712 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64de148b-68ea-4c76-b4e6-bce15aafd5d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:13.942016 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64de148b-68ea-4c76-b4e6-bce15aafd5d5 admin admin] PATH_INFO: `/v3/projects/50af65287f9d4a398c1cf93fe51f7b1c` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:13.943135 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64de148b-68ea-4c76-b4e6-bce15aafd5d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=50af65287f9d4a398c1cf93fe51f7b1c)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:13.945124 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64de148b-68ea-4c76-b4e6-bce15aafd5d5 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:14.004488 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-64de148b-68ea-4c76-b4e6-bce15aafd5d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50af65287f9d4a398c1cf93fe51f7b1c', 'request_id': 'req-64de148b-68ea-4c76-b4e6-bce15aafd5d5'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:14.011454 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-64de148b-68ea-4c76-b4e6-bce15aafd5d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50af65287f9d4a398c1cf93fe51f7b1c', 'request_id': 'req-64de148b-68ea-4c76-b4e6-bce15aafd5d5'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:14.020356 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-64de148b-68ea-4c76-b4e6-bce15aafd5d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50af65287f9d4a398c1cf93fe51f7b1c', 'request_id': 'req-64de148b-68ea-4c76-b4e6-bce15aafd5d5'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:14.032114 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:40:14.039103 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-64de148b-68ea-4c76-b4e6-bce15aafd5d5 admin admin] The token cache is being invalidate because project 50af65287f9d4a398c1cf93fe51f7b1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:40:14.039525 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-64de148b-68ea-4c76-b4e6-bce15aafd5d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:14.040277 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-64de148b-68ea-4c76-b4e6-bce15aafd5d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:14.043204 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1134/2262] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:40:13 2022] DELETE /identity/v3/projects/50af65287f9d4a398c1cf93fe51f7b1c => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:40:14.048211 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec5df69-8b31-4b28-a709-f7de5f419299 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:14.048711 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec5df69-8b31-4b28-a709-f7de5f419299 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:14.049006 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec5df69-8b31-4b28-a709-f7de5f419299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:14.072461 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-0ec5df69-8b31-4b28-a709-f7de5f419299 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:40:14.073742 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-0ec5df69-8b31-4b28-a709-f7de5f419299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:14.107773 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:40:14.151205 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1129/2263] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:40:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:40:14.158665 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafb9b78-d6aa-4750-a704-59bc6d76ad09 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:14.159470 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafb9b78-d6aa-4750-a704-59bc6d76ad09 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:14.160104 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafb9b78-d6aa-4750-a704-59bc6d76ad09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:14.188169 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-bafb9b78-d6aa-4750-a704-59bc6d76ad09 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:40:14.189682 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-bafb9b78-d6aa-4750-a704-59bc6d76ad09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:14.199597 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1135/2264] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:40:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 31 12:40:14.203776 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec5df69-8b31-4b28-a709-f7de5f419299 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:14.219143 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec5df69-8b31-4b28-a709-f7de5f419299 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:14.227280 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cdccd5-2bea-49e4-993d-5459e6db4948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:14.230108 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cdccd5-2bea-49e4-993d-5459e6db4948 admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:14.230791 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cdccd5-2bea-49e4-993d-5459e6db4948 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:14.231496 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cdccd5-2bea-49e4-993d-5459e6db4948 admin admin] PATH_INFO: `/v3/domains` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:14.232307 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cdccd5-2bea-49e4-993d-5459e6db4948 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:40:14.232723 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cdccd5-2bea-49e4-993d-5459e6db4948 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:14.234467 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cdccd5-2bea-49e4-993d-5459e6db4948 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:14.254255 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1130/2265] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Oct 31 12:40:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 31 12:40:14.255961 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafb9b78-d6aa-4750-a704-59bc6d76ad09 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:14.269700 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafb9b78-d6aa-4750-a704-59bc6d76ad09 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:14.277260 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6020ff0-e2e2-412a-872d-a5a1a1ef12b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:14.277260 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6020ff0-e2e2-412a-872d-a5a1a1ef12b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:14.277260 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6020ff0-e2e2-412a-872d-a5a1a1ef12b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:14.277714 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6020ff0-e2e2-412a-872d-a5a1a1ef12b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:14.278223 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6020ff0-e2e2-412a-872d-a5a1a1ef12b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:14.280783 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6020ff0-e2e2-412a-872d-a5a1a1ef12b1 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:14.324144 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1136/2266] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:40:14 2022] POST /identity/v3/projects => generated 373 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:40:14.324392 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cdccd5-2bea-49e4-993d-5459e6db4948 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:14.351465 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cdccd5-2bea-49e4-993d-5459e6db4948 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:14.445399 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaeadcf-0443-4032-bf19-ba3f75574f15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:14.447449 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaeadcf-0443-4032-bf19-ba3f75574f15 admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:14.447772 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaeadcf-0443-4032-bf19-ba3f75574f15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:14.448101 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaeadcf-0443-4032-bf19-ba3f75574f15 admin admin] PATH_INFO: `/v3/users` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:14.448740 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaeadcf-0443-4032-bf19-ba3f75574f15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:14.450807 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaeadcf-0443-4032-bf19-ba3f75574f15 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:14.479849 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1131/2267] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:40:14 2022] POST /identity/v3/users => generated 287 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:40:14.480978 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6020ff0-e2e2-412a-872d-a5a1a1ef12b1 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:14.491736 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6020ff0-e2e2-412a-872d-a5a1a1ef12b1 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:14.496606 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4a43dc-1393-44aa-81f3-02240420d8db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:14.497312 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4a43dc-1393-44aa-81f3-02240420d8db admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:14.497641 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4a43dc-1393-44aa-81f3-02240420d8db admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:14.497975 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4a43dc-1393-44aa-81f3-02240420d8db admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:14.502325 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4a43dc-1393-44aa-81f3-02240420d8db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:14.504040 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4a43dc-1393-44aa-81f3-02240420d8db admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:14.515294 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1137/2268] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:40:14 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 31 12:40:14.519737 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaeadcf-0443-4032-bf19-ba3f75574f15 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:14.530677 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaeadcf-0443-4032-bf19-ba3f75574f15 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:14.535345 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b9423e-621d-40f6-966b-fde4f0b47983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:14.535944 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b9423e-621d-40f6-966b-fde4f0b47983 admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:14.536223 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b9423e-621d-40f6-966b-fde4f0b47983 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:14.536494 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b9423e-621d-40f6-966b-fde4f0b47983 admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:14.537136 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b9423e-621d-40f6-966b-fde4f0b47983 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:14.538745 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b9423e-621d-40f6-966b-fde4f0b47983 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:14.553189 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1132/2269] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:40:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:40:14.554321 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4a43dc-1393-44aa-81f3-02240420d8db admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:14.564456 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4a43dc-1393-44aa-81f3-02240420d8db admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:14.569719 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b04f12-8a41-45d4-b4cd-fe4d34592dd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:14.570484 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b04f12-8a41-45d4-b4cd-fe4d34592dd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:14.570827 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b04f12-8a41-45d4-b4cd-fe4d34592dd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:14.571196 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b04f12-8a41-45d4-b4cd-fe4d34592dd1 admin admin] PATH_INFO: `/v3/projects/437b0ee5c1034635b0c7294f3e1ab629/users/7cec7f4586b04d46addee941a03a497c/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:14.586231 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b04f12-8a41-45d4-b4cd-fe4d34592dd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=437b0ee5c1034635b0c7294f3e1ab629, user_id=7cec7f4586b04d46addee941a03a497c, role_id=ca1a5695f8bd499aa0b8cedaeecd2cc6)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:14.589558 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b04f12-8a41-45d4-b4cd-fe4d34592dd1 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:14.602677 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1138/2270] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:40:14 2022] PUT /identity/v3/projects/437b0ee5c1034635b0c7294f3e1ab629/users/7cec7f4586b04d46addee941a03a497c/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:40:14.607986 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b145cfc9-ab58-4912-ad86-37cd90d484e6 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:14.608271 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b145cfc9-ab58-4912-ad86-37cd90d484e6 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:14.608538 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b145cfc9-ab58-4912-ad86-37cd90d484e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:14.627970 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-b145cfc9-ab58-4912-ad86-37cd90d484e6 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:40:14.629164 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-b145cfc9-ab58-4912-ad86-37cd90d484e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:14.700181 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1133/2271] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:40:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:40:14.709949 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b04f12-8a41-45d4-b4cd-fe4d34592dd1 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:14.712104 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-08b04f12-8a41-45d4-b4cd-fe4d34592dd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:14.785351 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b04f12-8a41-45d4-b4cd-fe4d34592dd1 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:14.789919 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc486dc-59e4-48c2-894b-00201c6501ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:14.791254 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc486dc-59e4-48c2-894b-00201c6501ab service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:14.791635 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc486dc-59e4-48c2-894b-00201c6501ab service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:14.791991 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc486dc-59e4-48c2-894b-00201c6501ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:14.805162 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc486dc-59e4-48c2-894b-00201c6501ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:14.806705 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc486dc-59e4-48c2-894b-00201c6501ab service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:14.817490 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1139/2272] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:40:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:40:15.443108 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d4dffd-1f1c-4ef5-a311-0164cece4458 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:15.443426 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d4dffd-1f1c-4ef5-a311-0164cece4458 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:15.443696 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d4dffd-1f1c-4ef5-a311-0164cece4458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:15.471991 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-27d4dffd-1f1c-4ef5-a311-0164cece4458 None None] MFA Rules not processed for user `7cec7f4586b04d46addee941a03a497c`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:40:15.473196 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-27d4dffd-1f1c-4ef5-a311-0164cece4458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:15.505790 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:40:15.546320 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1134/2273] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:40:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:40:15.550625 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc486dc-59e4-48c2-894b-00201c6501ab service neutron] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:15.561230 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc486dc-59e4-48c2-894b-00201c6501ab service neutron] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:15.565979 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b556cb0-47f6-47c6-9d2a-cdc6a6704b65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:15.566574 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b556cb0-47f6-47c6-9d2a-cdc6a6704b65 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:15.566849 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b556cb0-47f6-47c6-9d2a-cdc6a6704b65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:15.567121 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b556cb0-47f6-47c6-9d2a-cdc6a6704b65 admin admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:15.567769 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b556cb0-47f6-47c6-9d2a-cdc6a6704b65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:15.569678 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b556cb0-47f6-47c6-9d2a-cdc6a6704b65 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:15.598519 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1140/2274] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:40:15 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:40:15.602508 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d4dffd-1f1c-4ef5-a311-0164cece4458 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:15.614004 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d4dffd-1f1c-4ef5-a311-0164cece4458 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:15.671374 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad58e125-3cfe-4375-81dc-bc728ee23100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:15.671980 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad58e125-3cfe-4375-81dc-bc728ee23100 admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:15.672257 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad58e125-3cfe-4375-81dc-bc728ee23100 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:15.672525 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad58e125-3cfe-4375-81dc-bc728ee23100 admin admin] PATH_INFO: `/v3/users` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:15.673147 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad58e125-3cfe-4375-81dc-bc728ee23100 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:15.675059 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad58e125-3cfe-4375-81dc-bc728ee23100 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:15.696090 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1135/2275] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:40:15 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 31 12:40:15.700591 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b556cb0-47f6-47c6-9d2a-cdc6a6704b65 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:15.710646 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b556cb0-47f6-47c6-9d2a-cdc6a6704b65 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:15.715740 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee1ccb3-79c3-4244-9412-5f01b2f7e8a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:15.716316 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee1ccb3-79c3-4244-9412-5f01b2f7e8a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:15.716588 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee1ccb3-79c3-4244-9412-5f01b2f7e8a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:15.716853 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee1ccb3-79c3-4244-9412-5f01b2f7e8a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:15.717479 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee1ccb3-79c3-4244-9412-5f01b2f7e8a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:15.719042 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee1ccb3-79c3-4244-9412-5f01b2f7e8a6 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:15.729339 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1141/2276] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:40:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:40:15.733268 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad58e125-3cfe-4375-81dc-bc728ee23100 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:15.744428 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad58e125-3cfe-4375-81dc-bc728ee23100 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:15.749061 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e71e8c0-bd5c-46d2-86d2-38441a693837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:15.749722 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e71e8c0-bd5c-46d2-86d2-38441a693837 admin admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:15.750009 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e71e8c0-bd5c-46d2-86d2-38441a693837 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:15.750282 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e71e8c0-bd5c-46d2-86d2-38441a693837 admin admin] PATH_INFO: `/v3/projects/e2b1110b90b84855b66acd04e23c70de/users/7e302c270ece47a895a0fc6d377b929d/roles/176c07b4fe3b4537bdfeb4be21b379c5` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:15.765637 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e71e8c0-bd5c-46d2-86d2-38441a693837 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2b1110b90b84855b66acd04e23c70de, user_id=7e302c270ece47a895a0fc6d377b929d, role_id=176c07b4fe3b4537bdfeb4be21b379c5)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:15.769018 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e71e8c0-bd5c-46d2-86d2-38441a693837 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:15.781305 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1136/2277] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:40:15 2022] PUT /identity/v3/projects/e2b1110b90b84855b66acd04e23c70de/users/7e302c270ece47a895a0fc6d377b929d/roles/176c07b4fe3b4537bdfeb4be21b379c5 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:40:15.785170 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee1ccb3-79c3-4244-9412-5f01b2f7e8a6 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:15.796945 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee1ccb3-79c3-4244-9412-5f01b2f7e8a6 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:15.852329 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1744a477-6d08-4ede-bda6-18e4fd57f1ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:15.853258 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1744a477-6d08-4ede-bda6-18e4fd57f1ae admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:15.853539 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1744a477-6d08-4ede-bda6-18e4fd57f1ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:15.853808 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1744a477-6d08-4ede-bda6-18e4fd57f1ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:15.854436 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1744a477-6d08-4ede-bda6-18e4fd57f1ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:15.856040 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1744a477-6d08-4ede-bda6-18e4fd57f1ae admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:15.861331 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:40:15.867337 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1142/2278] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:40:15 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:40:15.871660 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e71e8c0-bd5c-46d2-86d2-38441a693837 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:15.882596 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e71e8c0-bd5c-46d2-86d2-38441a693837 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:15.888944 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb86fa72-153d-4fea-b12b-96c5f93bec0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:15.889649 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb86fa72-153d-4fea-b12b-96c5f93bec0f admin admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:15.889932 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb86fa72-153d-4fea-b12b-96c5f93bec0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:15.890207 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb86fa72-153d-4fea-b12b-96c5f93bec0f admin admin] PATH_INFO: `/v3/projects/e2b1110b90b84855b66acd04e23c70de/users/7e302c270ece47a895a0fc6d377b929d/roles/176c07b4fe3b4537bdfeb4be21b379c5` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:15.892359 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb86fa72-153d-4fea-b12b-96c5f93bec0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2b1110b90b84855b66acd04e23c70de, user_id=7e302c270ece47a895a0fc6d377b929d, role_id=176c07b4fe3b4537bdfeb4be21b379c5)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:15.895282 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb86fa72-153d-4fea-b12b-96c5f93bec0f admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:15.907089 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1137/2279] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:40:15 2022] PUT /identity/v3/projects/e2b1110b90b84855b66acd04e23c70de/users/7e302c270ece47a895a0fc6d377b929d/roles/176c07b4fe3b4537bdfeb4be21b379c5 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:40:16.532059 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e296f6-57ef-42f3-bba3-69fea69e6278 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:16.532347 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e296f6-57ef-42f3-bba3-69fea69e6278 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:16.532617 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e296f6-57ef-42f3-bba3-69fea69e6278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:16.559753 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-b1e296f6-57ef-42f3-bba3-69fea69e6278 None None] MFA Rules not processed for user `7e302c270ece47a895a0fc6d377b929d`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:40:16.560964 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-b1e296f6-57ef-42f3-bba3-69fea69e6278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:16.652202 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1143/2280] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:40:16 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:40:16.659036 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb86fa72-153d-4fea-b12b-96c5f93bec0f admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:16.660827 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-fb86fa72-153d-4fea-b12b-96c5f93bec0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:16.717750 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb86fa72-153d-4fea-b12b-96c5f93bec0f admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:16.721235 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b3b357-5abc-4927-9c4f-d617f1766eaf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d6d7ea077d49e8a66c6c6762499702', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:16.721916 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b3b357-5abc-4927-9c4f-d617f1766eaf service cinder] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:16.722111 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b3b357-5abc-4927-9c4f-d617f1766eaf service cinder] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:16.722380 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b3b357-5abc-4927-9c4f-d617f1766eaf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:16.760570 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:40:16.784185 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b3b357-5abc-4927-9c4f-d617f1766eaf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:16.785587 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b3b357-5abc-4927-9c4f-d617f1766eaf service cinder] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:16.795769 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1138/2281] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:40:16 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:40:17.364166 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e296f6-57ef-42f3-bba3-69fea69e6278 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:17.420809 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e296f6-57ef-42f3-bba3-69fea69e6278 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:17.424375 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5873a432-9beb-4ec9-8c9f-f9c6858810b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:17.424978 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5873a432-9beb-4ec9-8c9f-f9c6858810b4 service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:17.425257 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5873a432-9beb-4ec9-8c9f-f9c6858810b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:17.425530 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5873a432-9beb-4ec9-8c9f-f9c6858810b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:17.427292 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-5873a432-9beb-4ec9-8c9f-f9c6858810b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:17.488921 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5873a432-9beb-4ec9-8c9f-f9c6858810b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:17.490332 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5873a432-9beb-4ec9-8c9f-f9c6858810b4 service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:17.500410 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1144/2282] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:40:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 31 12:40:18.155238 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b3b357-5abc-4927-9c4f-d617f1766eaf service cinder] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:18.156736 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-17b3b357-5abc-4927-9c4f-d617f1766eaf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:18.169711 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b3b357-5abc-4927-9c4f-d617f1766eaf service cinder] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:18.173795 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b305d11-c44a-4e82-b3e4-74df71f84708 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d6d7ea077d49e8a66c6c6762499702', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:18.174406 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b305d11-c44a-4e82-b3e4-74df71f84708 service cinder] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:18.174723 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b305d11-c44a-4e82-b3e4-74df71f84708 service cinder] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:18.174980 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b305d11-c44a-4e82-b3e4-74df71f84708 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:18.186942 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b305d11-c44a-4e82-b3e4-74df71f84708 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:18.188739 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b305d11-c44a-4e82-b3e4-74df71f84708 service cinder] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:18.199766 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1139/2283] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:40:18 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:40:21.018389 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5873a432-9beb-4ec9-8c9f-f9c6858810b4 service neutron] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:21.019896 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-5873a432-9beb-4ec9-8c9f-f9c6858810b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:21.035233 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5873a432-9beb-4ec9-8c9f-f9c6858810b4 service neutron] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:21.039414 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de325af9-e2e4-4594-8e16-418c1e24fedb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:21.040047 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de325af9-e2e4-4594-8e16-418c1e24fedb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:21.040495 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de325af9-e2e4-4594-8e16-418c1e24fedb admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:21.040938 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de325af9-e2e4-4594-8e16-418c1e24fedb admin admin] PATH_INFO: `/v3/users/a6d1ace68aa84499a4929da60f3d71bf` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:21.042063 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de325af9-e2e4-4594-8e16-418c1e24fedb admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6d1ace68aa84499a4929da60f3d71bf)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:21.044017 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de325af9-e2e4-4594-8e16-418c1e24fedb admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:21.078708 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:40:21.093957 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-de325af9-e2e4-4594-8e16-418c1e24fedb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6d1ace68aa84499a4929da60f3d71bf'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:21.101438 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-de325af9-e2e4-4594-8e16-418c1e24fedb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6d1ace68aa84499a4929da60f3d71bf'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:21.102473 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-de325af9-e2e4-4594-8e16-418c1e24fedb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6d1ace68aa84499a4929da60f3d71bf'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:21.103191 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-de325af9-e2e4-4594-8e16-418c1e24fedb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6d1ace68aa84499a4929da60f3d71bf'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:21.115273 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-de325af9-e2e4-4594-8e16-418c1e24fedb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6d1ace68aa84499a4929da60f3d71bf'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:21.127277 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1145/2284] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:40:21 2022] DELETE /identity/v3/users/a6d1ace68aa84499a4929da60f3d71bf => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:40:21.302591 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b305d11-c44a-4e82-b3e4-74df71f84708 service cinder] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:21.313962 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b305d11-c44a-4e82-b3e4-74df71f84708 service cinder] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:21.375108 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a692a4b5-2cc5-41b7-b140-abd2e7e96350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:21.375770 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a692a4b5-2cc5-41b7-b140-abd2e7e96350 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:21.376053 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a692a4b5-2cc5-41b7-b140-abd2e7e96350 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:21.376324 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a692a4b5-2cc5-41b7-b140-abd2e7e96350 admin admin] PATH_INFO: `/v3/projects/de0eb090bf44403983e0cbb99e038321` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:21.377416 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a692a4b5-2cc5-41b7-b140-abd2e7e96350 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de0eb090bf44403983e0cbb99e038321)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:21.379402 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a692a4b5-2cc5-41b7-b140-abd2e7e96350 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:21.426446 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-a692a4b5-2cc5-41b7-b140-abd2e7e96350 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de0eb090bf44403983e0cbb99e038321', 'request_id': 'req-a692a4b5-2cc5-41b7-b140-abd2e7e96350'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:21.432883 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-a692a4b5-2cc5-41b7-b140-abd2e7e96350 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de0eb090bf44403983e0cbb99e038321', 'request_id': 'req-a692a4b5-2cc5-41b7-b140-abd2e7e96350'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:21.440370 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-a692a4b5-2cc5-41b7-b140-abd2e7e96350 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de0eb090bf44403983e0cbb99e038321', 'request_id': 'req-a692a4b5-2cc5-41b7-b140-abd2e7e96350'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:21.452242 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-a692a4b5-2cc5-41b7-b140-abd2e7e96350 admin admin] The token cache is being invalidate because project de0eb090bf44403983e0cbb99e038321 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:40:21.452525 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-a692a4b5-2cc5-41b7-b140-abd2e7e96350 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:21.453155 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-a692a4b5-2cc5-41b7-b140-abd2e7e96350 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:21.454662 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1140/2285] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:40:21 2022] DELETE /identity/v3/projects/de0eb090bf44403983e0cbb99e038321 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 31 12:40:21.459679 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe68d9e-9069-4ae9-b6f3-fd1c2c725236 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:21.459972 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe68d9e-9069-4ae9-b6f3-fd1c2c725236 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:21.460248 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe68d9e-9069-4ae9-b6f3-fd1c2c725236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:21.480673 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-ffe68d9e-9069-4ae9-b6f3-fd1c2c725236 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:40:21.481802 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-ffe68d9e-9069-4ae9-b6f3-fd1c2c725236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:21.541322 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1146/2286] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:40:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:40:21.549009 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2625f586-ba47-407f-9ef0-fc2e9f0fefd5 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:21.549298 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2625f586-ba47-407f-9ef0-fc2e9f0fefd5 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:21.549576 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2625f586-ba47-407f-9ef0-fc2e9f0fefd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:21.572725 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-2625f586-ba47-407f-9ef0-fc2e9f0fefd5 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:40:21.573922 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-2625f586-ba47-407f-9ef0-fc2e9f0fefd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:21.581881 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1141/2287] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:40:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:40:21.586353 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe68d9e-9069-4ae9-b6f3-fd1c2c725236 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:21.596780 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe68d9e-9069-4ae9-b6f3-fd1c2c725236 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:21.601495 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6243e0e4-92b9-45bd-8315-721ec8ecef26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:21.602094 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6243e0e4-92b9-45bd-8315-721ec8ecef26 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:21.602372 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6243e0e4-92b9-45bd-8315-721ec8ecef26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:21.602667 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6243e0e4-92b9-45bd-8315-721ec8ecef26 admin admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:21.603977 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6243e0e4-92b9-45bd-8315-721ec8ecef26 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:40:21.604282 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6243e0e4-92b9-45bd-8315-721ec8ecef26 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:21.605856 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6243e0e4-92b9-45bd-8315-721ec8ecef26 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:21.617859 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1147/2288] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Oct 31 12:40:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 31 12:40:21.622828 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2625f586-ba47-407f-9ef0-fc2e9f0fefd5 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:21.632092 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2625f586-ba47-407f-9ef0-fc2e9f0fefd5 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:21.637475 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57381ee-7d65-4501-b22c-e5a346417294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:21.638075 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57381ee-7d65-4501-b22c-e5a346417294 admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:21.638385 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57381ee-7d65-4501-b22c-e5a346417294 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:21.638659 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57381ee-7d65-4501-b22c-e5a346417294 admin admin] PATH_INFO: `/v3/projects` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:21.639325 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57381ee-7d65-4501-b22c-e5a346417294 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:21.641304 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57381ee-7d65-4501-b22c-e5a346417294 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:21.675072 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1142/2289] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:40:21 2022] POST /identity/v3/projects => generated 365 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:40:21.678915 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6243e0e4-92b9-45bd-8315-721ec8ecef26 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:21.689690 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6243e0e4-92b9-45bd-8315-721ec8ecef26 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:21.745470 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf858c6-7a7c-4307-858c-d530df40db04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:21.746003 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf858c6-7a7c-4307-858c-d530df40db04 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:21.746283 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf858c6-7a7c-4307-858c-d530df40db04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:21.746555 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf858c6-7a7c-4307-858c-d530df40db04 admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:21.747205 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf858c6-7a7c-4307-858c-d530df40db04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:21.749126 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf858c6-7a7c-4307-858c-d530df40db04 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:21.769892 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1148/2290] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:40:21 2022] POST /identity/v3/users => generated 283 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:40:21.774461 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57381ee-7d65-4501-b22c-e5a346417294 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:21.784918 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57381ee-7d65-4501-b22c-e5a346417294 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:21.795878 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af6bb08-3f3e-4f62-a76e-15f091fa6e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:21.796492 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af6bb08-3f3e-4f62-a76e-15f091fa6e0e admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:21.796763 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af6bb08-3f3e-4f62-a76e-15f091fa6e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:21.797028 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af6bb08-3f3e-4f62-a76e-15f091fa6e0e admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:21.797663 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af6bb08-3f3e-4f62-a76e-15f091fa6e0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:21.799270 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af6bb08-3f3e-4f62-a76e-15f091fa6e0e admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:21.809625 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1143/2291] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:40:21 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:40:21.813822 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf858c6-7a7c-4307-858c-d530df40db04 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:21.824263 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf858c6-7a7c-4307-858c-d530df40db04 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:21.828753 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335cc3ca-6a5c-4751-838f-0d3b2b00be61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:21.829344 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335cc3ca-6a5c-4751-838f-0d3b2b00be61 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:21.829623 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335cc3ca-6a5c-4751-838f-0d3b2b00be61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:21.829892 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335cc3ca-6a5c-4751-838f-0d3b2b00be61 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:21.830708 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335cc3ca-6a5c-4751-838f-0d3b2b00be61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:21.832326 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335cc3ca-6a5c-4751-838f-0d3b2b00be61 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:21.842891 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1149/2292] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:40:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:40:21.846818 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af6bb08-3f3e-4f62-a76e-15f091fa6e0e admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:21.857029 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af6bb08-3f3e-4f62-a76e-15f091fa6e0e admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:21.861474 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ee5198-1963-4866-8d55-8576fc4350af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:21.862101 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ee5198-1963-4866-8d55-8576fc4350af admin admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:21.862373 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ee5198-1963-4866-8d55-8576fc4350af admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:21.862639 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ee5198-1963-4866-8d55-8576fc4350af admin admin] PATH_INFO: `/v3/projects/68c14413792e43069c4cefb5879dea99/users/53b3eefefe2e40b2901d094746590b21/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:21.873960 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:40:21.878485 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ee5198-1963-4866-8d55-8576fc4350af admin admin] RBAC: Authorizing `identity:create_grant(project_id=68c14413792e43069c4cefb5879dea99, user_id=53b3eefefe2e40b2901d094746590b21, role_id=ca1a5695f8bd499aa0b8cedaeecd2cc6)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:21.881671 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ee5198-1963-4866-8d55-8576fc4350af admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:21.895306 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1144/2293] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:40:21 2022] PUT /identity/v3/projects/68c14413792e43069c4cefb5879dea99/users/53b3eefefe2e40b2901d094746590b21/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:40:21.905804 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22fa658-cd2e-49d5-a1ed-566ada944494 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:21.906092 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22fa658-cd2e-49d5-a1ed-566ada944494 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:21.906394 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22fa658-cd2e-49d5-a1ed-566ada944494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:21.934886 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:40:21.938548 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-e22fa658-cd2e-49d5-a1ed-566ada944494 None None] MFA Rules not processed for user `53b3eefefe2e40b2901d094746590b21`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:40:21.940452 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-e22fa658-cd2e-49d5-a1ed-566ada944494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:22.012402 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1150/2294] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:40:21 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:40:22.025668 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ee5198-1963-4866-8d55-8576fc4350af admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:22.027610 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-18ee5198-1963-4866-8d55-8576fc4350af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:22.088292 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ee5198-1963-4866-8d55-8576fc4350af admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:22.092079 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d1d721-0de6-419b-9935-26bb2e86ab92 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d6d7ea077d49e8a66c6c6762499702', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:22.092715 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d1d721-0de6-419b-9935-26bb2e86ab92 service cinder] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:22.092994 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d1d721-0de6-419b-9935-26bb2e86ab92 service cinder] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:22.093267 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d1d721-0de6-419b-9935-26bb2e86ab92 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:22.101679 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d1d721-0de6-419b-9935-26bb2e86ab92 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:22.103227 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d1d721-0de6-419b-9935-26bb2e86ab92 service cinder] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:22.112244 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1145/2295] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:40:22 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:40:32.009207 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=26 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:40:32.108256 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:40:34.774968 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22fa658-cd2e-49d5-a1ed-566ada944494 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:34.836859 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22fa658-cd2e-49d5-a1ed-566ada944494 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:34.841387 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3efd719-9d72-4def-a4cc-b5b80cddd7a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:34.842007 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3efd719-9d72-4def-a4cc-b5b80cddd7a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:34.842280 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3efd719-9d72-4def-a4cc-b5b80cddd7a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:34.842547 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3efd719-9d72-4def-a4cc-b5b80cddd7a0 admin admin] PATH_INFO: `/v3/users/53b3eefefe2e40b2901d094746590b21` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:34.843675 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3efd719-9d72-4def-a4cc-b5b80cddd7a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53b3eefefe2e40b2901d094746590b21)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:34.845545 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3efd719-9d72-4def-a4cc-b5b80cddd7a0 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:34.897076 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-b3efd719-9d72-4def-a4cc-b5b80cddd7a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53b3eefefe2e40b2901d094746590b21'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:34.904868 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-b3efd719-9d72-4def-a4cc-b5b80cddd7a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53b3eefefe2e40b2901d094746590b21'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:34.905705 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-b3efd719-9d72-4def-a4cc-b5b80cddd7a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53b3eefefe2e40b2901d094746590b21'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:34.906319 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-b3efd719-9d72-4def-a4cc-b5b80cddd7a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53b3eefefe2e40b2901d094746590b21'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:34.921081 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-b3efd719-9d72-4def-a4cc-b5b80cddd7a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53b3eefefe2e40b2901d094746590b21'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:34.935082 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1151/2296] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:40:34 2022] DELETE /identity/v3/users/53b3eefefe2e40b2901d094746590b21 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:40:34.939354 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab7c67a-cd64-46e1-a6e3-da298ba869ad None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:34.939811 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab7c67a-cd64-46e1-a6e3-da298ba869ad None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:34.939922 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab7c67a-cd64-46e1-a6e3-da298ba869ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:34.959152 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-bab7c67a-cd64-46e1-a6e3-da298ba869ad None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:40:34.960308 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-bab7c67a-cd64-46e1-a6e3-da298ba869ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:35.020832 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1146/2297] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:40:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 31 12:40:35.030727 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3efd719-9d72-4def-a4cc-b5b80cddd7a0 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:35.032264 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-b3efd719-9d72-4def-a4cc-b5b80cddd7a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:35.093343 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3efd719-9d72-4def-a4cc-b5b80cddd7a0 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:35.097261 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288fab09-bea1-46ba-9b65-c6e1ceedb3c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:35.098004 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288fab09-bea1-46ba-9b65-c6e1ceedb3c6 service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:35.098499 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288fab09-bea1-46ba-9b65-c6e1ceedb3c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:35.098839 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288fab09-bea1-46ba-9b65-c6e1ceedb3c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:35.112048 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288fab09-bea1-46ba-9b65-c6e1ceedb3c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:35.113518 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288fab09-bea1-46ba-9b65-c6e1ceedb3c6 service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:35.124937 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1152/2298] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:40:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:40:35.401921 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab7c67a-cd64-46e1-a6e3-da298ba869ad None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:35.403583 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-bab7c67a-cd64-46e1-a6e3-da298ba869ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:35.417666 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab7c67a-cd64-46e1-a6e3-da298ba869ad None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:35.422803 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78774ba5-6287-4a6c-8c46-8e6c9e4c5db8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:35.423521 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78774ba5-6287-4a6c-8c46-8e6c9e4c5db8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:35.423801 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78774ba5-6287-4a6c-8c46-8e6c9e4c5db8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:35.424146 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78774ba5-6287-4a6c-8c46-8e6c9e4c5db8 admin admin] PATH_INFO: `/v3/projects/68c14413792e43069c4cefb5879dea99` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:35.425263 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78774ba5-6287-4a6c-8c46-8e6c9e4c5db8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68c14413792e43069c4cefb5879dea99)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:35.427322 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78774ba5-6287-4a6c-8c46-8e6c9e4c5db8 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:35.477079 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-78774ba5-6287-4a6c-8c46-8e6c9e4c5db8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68c14413792e43069c4cefb5879dea99', 'request_id': 'req-78774ba5-6287-4a6c-8c46-8e6c9e4c5db8'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:35.483268 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-78774ba5-6287-4a6c-8c46-8e6c9e4c5db8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68c14413792e43069c4cefb5879dea99', 'request_id': 'req-78774ba5-6287-4a6c-8c46-8e6c9e4c5db8'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:35.491517 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-78774ba5-6287-4a6c-8c46-8e6c9e4c5db8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68c14413792e43069c4cefb5879dea99', 'request_id': 'req-78774ba5-6287-4a6c-8c46-8e6c9e4c5db8'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:35.503141 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-78774ba5-6287-4a6c-8c46-8e6c9e4c5db8 admin admin] The token cache is being invalidate because project 68c14413792e43069c4cefb5879dea99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:40:35.503457 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-78774ba5-6287-4a6c-8c46-8e6c9e4c5db8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:35.504079 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-78774ba5-6287-4a6c-8c46-8e6c9e4c5db8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:35.505556 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1147/2299] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:40:35 2022] DELETE /identity/v3/projects/68c14413792e43069c4cefb5879dea99 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:40:35.510362 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e7e887-6eba-4d25-8aaa-617290ae8435 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:35.510699 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e7e887-6eba-4d25-8aaa-617290ae8435 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:35.511103 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e7e887-6eba-4d25-8aaa-617290ae8435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:35.531991 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-61e7e887-6eba-4d25-8aaa-617290ae8435 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:40:35.533193 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-61e7e887-6eba-4d25-8aaa-617290ae8435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:35.552190 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:40:35.603724 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1153/2300] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:40:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:40:35.611523 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e21efc-0ffe-4af2-9913-da8755884a46 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:35.611816 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e21efc-0ffe-4af2-9913-da8755884a46 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:35.612070 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e21efc-0ffe-4af2-9913-da8755884a46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:35.633043 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-97e21efc-0ffe-4af2-9913-da8755884a46 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:40:35.634366 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-97e21efc-0ffe-4af2-9913-da8755884a46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:35.643358 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1148/2301] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:40:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:40:35.647690 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e7e887-6eba-4d25-8aaa-617290ae8435 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:35.657486 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e7e887-6eba-4d25-8aaa-617290ae8435 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:35.662078 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5f4de4-87d7-4422-8470-993d1359abb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:35.662789 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5f4de4-87d7-4422-8470-993d1359abb4 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:35.663068 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5f4de4-87d7-4422-8470-993d1359abb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:35.663398 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5f4de4-87d7-4422-8470-993d1359abb4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:35.664037 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5f4de4-87d7-4422-8470-993d1359abb4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:40:35.664343 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5f4de4-87d7-4422-8470-993d1359abb4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:35.665952 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5f4de4-87d7-4422-8470-993d1359abb4 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:35.681602 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1154/2302] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Oct 31 12:40:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:40:35.683320 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e21efc-0ffe-4af2-9913-da8755884a46 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:35.692772 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e21efc-0ffe-4af2-9913-da8755884a46 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:35.697600 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687b940d-2daf-4f0a-9d37-1b9abd878dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:35.698305 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687b940d-2daf-4f0a-9d37-1b9abd878dcb admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:35.698687 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687b940d-2daf-4f0a-9d37-1b9abd878dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:35.698955 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687b940d-2daf-4f0a-9d37-1b9abd878dcb admin admin] PATH_INFO: `/v3/projects` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:35.699683 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687b940d-2daf-4f0a-9d37-1b9abd878dcb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:35.701563 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687b940d-2daf-4f0a-9d37-1b9abd878dcb admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:35.733334 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1149/2303] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:40:35 2022] POST /identity/v3/projects => generated 369 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 31 12:40:35.734709 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5f4de4-87d7-4422-8470-993d1359abb4 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:35.744779 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5f4de4-87d7-4422-8470-993d1359abb4 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:35.805817 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92080bda-3e3e-4926-b0b5-278adcdac2ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:35.806560 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92080bda-3e3e-4926-b0b5-278adcdac2ec admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:35.806849 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92080bda-3e3e-4926-b0b5-278adcdac2ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:35.807123 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92080bda-3e3e-4926-b0b5-278adcdac2ec admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:35.808301 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92080bda-3e3e-4926-b0b5-278adcdac2ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:35.810446 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92080bda-3e3e-4926-b0b5-278adcdac2ec admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:35.831850 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1155/2304] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:40:35 2022] POST /identity/v3/users => generated 285 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 31 12:40:35.835858 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687b940d-2daf-4f0a-9d37-1b9abd878dcb admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:35.846166 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687b940d-2daf-4f0a-9d37-1b9abd878dcb admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:35.850931 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce20129c-34f9-4720-9eff-6be3b9375f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:35.851570 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce20129c-34f9-4720-9eff-6be3b9375f5c admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:35.851846 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce20129c-34f9-4720-9eff-6be3b9375f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:35.852115 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce20129c-34f9-4720-9eff-6be3b9375f5c admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:35.852932 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce20129c-34f9-4720-9eff-6be3b9375f5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:35.854851 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce20129c-34f9-4720-9eff-6be3b9375f5c admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:35.866018 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1150/2305] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:40:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:40:35.870153 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92080bda-3e3e-4926-b0b5-278adcdac2ec admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:35.880732 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92080bda-3e3e-4926-b0b5-278adcdac2ec admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:35.885413 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe36ba8-77b9-4047-89a9-84d39b0df333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:35.886485 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe36ba8-77b9-4047-89a9-84d39b0df333 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:35.886768 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe36ba8-77b9-4047-89a9-84d39b0df333 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:35.887023 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe36ba8-77b9-4047-89a9-84d39b0df333 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:35.891687 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe36ba8-77b9-4047-89a9-84d39b0df333 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:35.893259 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe36ba8-77b9-4047-89a9-84d39b0df333 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:35.905809 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1156/2306] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:40:35 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:40:35.910144 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce20129c-34f9-4720-9eff-6be3b9375f5c admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:35.920450 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce20129c-34f9-4720-9eff-6be3b9375f5c admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:35.926660 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a68ab4-643c-4c04-8cce-5c8c06ebeea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:35.927434 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a68ab4-643c-4c04-8cce-5c8c06ebeea3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:35.927877 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a68ab4-643c-4c04-8cce-5c8c06ebeea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:35.928728 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a68ab4-643c-4c04-8cce-5c8c06ebeea3 admin admin] PATH_INFO: `/v3/projects/8e32724626464d01bd7f866693de0a42/users/9f58c4ba32254b77b33c6bcc26d8852c/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:35.946775 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a68ab4-643c-4c04-8cce-5c8c06ebeea3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e32724626464d01bd7f866693de0a42, user_id=9f58c4ba32254b77b33c6bcc26d8852c, role_id=ca1a5695f8bd499aa0b8cedaeecd2cc6)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:35.950663 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a68ab4-643c-4c04-8cce-5c8c06ebeea3 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:35.964011 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1151/2307] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:40:35 2022] PUT /identity/v3/projects/8e32724626464d01bd7f866693de0a42/users/9f58c4ba32254b77b33c6bcc26d8852c/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 31 12:40:35.975411 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63a4913-17f0-4979-b576-a95185f51a5b None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:35.975917 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63a4913-17f0-4979-b576-a95185f51a5b None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:35.976184 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63a4913-17f0-4979-b576-a95185f51a5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:36.005178 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-b63a4913-17f0-4979-b576-a95185f51a5b None None] MFA Rules not processed for user `9f58c4ba32254b77b33c6bcc26d8852c`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:40:36.006358 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-b63a4913-17f0-4979-b576-a95185f51a5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:36.080583 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1157/2308] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:40:35 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:40:36.085021 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a68ab4-643c-4c04-8cce-5c8c06ebeea3 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:36.108498 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:40:36.154455 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a68ab4-643c-4c04-8cce-5c8c06ebeea3 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:36.159452 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab2d63c-2463-4f66-b823-09a39275c2cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:36.160060 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab2d63c-2463-4f66-b823-09a39275c2cc admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:36.160339 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab2d63c-2463-4f66-b823-09a39275c2cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:36.160606 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab2d63c-2463-4f66-b823-09a39275c2cc admin admin] PATH_INFO: `/v3/projects` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:36.161248 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab2d63c-2463-4f66-b823-09a39275c2cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:36.163387 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab2d63c-2463-4f66-b823-09a39275c2cc admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:36.196155 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1152/2309] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:40:36 2022] POST /identity/v3/projects => generated 367 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:40:36.197552 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63a4913-17f0-4979-b576-a95185f51a5b None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:36.208593 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63a4913-17f0-4979-b576-a95185f51a5b None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:36.225682 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:40:36.290802 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d558e0e5-6fe0-47c5-9c36-181bf5b39af2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:36.292447 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d558e0e5-6fe0-47c5-9c36-181bf5b39af2 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:36.293189 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d558e0e5-6fe0-47c5-9c36-181bf5b39af2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:36.293189 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d558e0e5-6fe0-47c5-9c36-181bf5b39af2 admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:36.293652 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d558e0e5-6fe0-47c5-9c36-181bf5b39af2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:36.295607 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d558e0e5-6fe0-47c5-9c36-181bf5b39af2 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:36.320040 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1158/2310] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:40:36 2022] POST /identity/v3/users => generated 290 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:40:36.324031 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab2d63c-2463-4f66-b823-09a39275c2cc admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:36.334997 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab2d63c-2463-4f66-b823-09a39275c2cc admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:36.342656 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c78726-46e7-4686-bec8-ebf8efca3be3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:36.343426 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c78726-46e7-4686-bec8-ebf8efca3be3 admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:36.343808 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c78726-46e7-4686-bec8-ebf8efca3be3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:36.344368 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c78726-46e7-4686-bec8-ebf8efca3be3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:36.344953 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c78726-46e7-4686-bec8-ebf8efca3be3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:36.347123 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c78726-46e7-4686-bec8-ebf8efca3be3 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:36.365004 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1153/2311] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:40:36 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:40:36.368111 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d558e0e5-6fe0-47c5-9c36-181bf5b39af2 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:36.379254 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d558e0e5-6fe0-47c5-9c36-181bf5b39af2 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:36.384298 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab2c92e-6a17-4700-986c-0b2a4ef8ffc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:36.384964 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab2c92e-6a17-4700-986c-0b2a4ef8ffc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:36.385245 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab2c92e-6a17-4700-986c-0b2a4ef8ffc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:36.385548 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab2c92e-6a17-4700-986c-0b2a4ef8ffc5 admin admin] PATH_INFO: `/v3/projects/a8f8077e970a40d0a3df6250cad57c4e/users/7d65ca13f4d44238ae6bfd4ee5786484/roles/176c07b4fe3b4537bdfeb4be21b379c5` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:36.405375 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab2c92e-6a17-4700-986c-0b2a4ef8ffc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8f8077e970a40d0a3df6250cad57c4e, user_id=7d65ca13f4d44238ae6bfd4ee5786484, role_id=176c07b4fe3b4537bdfeb4be21b379c5)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:36.410868 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab2c92e-6a17-4700-986c-0b2a4ef8ffc5 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:36.427087 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1159/2312] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:40:36 2022] PUT /identity/v3/projects/a8f8077e970a40d0a3df6250cad57c4e/users/7d65ca13f4d44238ae6bfd4ee5786484/roles/176c07b4fe3b4537bdfeb4be21b379c5 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:40:36.430985 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c78726-46e7-4686-bec8-ebf8efca3be3 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:36.446845 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c78726-46e7-4686-bec8-ebf8efca3be3 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:36.518999 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4025a6e5-af13-484c-841e-a92739533eec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:36.519794 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4025a6e5-af13-484c-841e-a92739533eec admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:36.520178 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4025a6e5-af13-484c-841e-a92739533eec admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:36.520552 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4025a6e5-af13-484c-841e-a92739533eec admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:36.521385 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4025a6e5-af13-484c-841e-a92739533eec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:36.523138 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4025a6e5-af13-484c-841e-a92739533eec admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:36.538281 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1154/2313] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:40:36 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 31 12:40:36.539696 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab2c92e-6a17-4700-986c-0b2a4ef8ffc5 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:36.551380 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab2c92e-6a17-4700-986c-0b2a4ef8ffc5 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:36.556658 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4d0ed5-9ed0-4b78-a7ab-8ad71aa74a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:36.557318 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4d0ed5-9ed0-4b78-a7ab-8ad71aa74a50 admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:36.558112 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4d0ed5-9ed0-4b78-a7ab-8ad71aa74a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:36.558382 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4d0ed5-9ed0-4b78-a7ab-8ad71aa74a50 admin admin] PATH_INFO: `/v3/projects/a8f8077e970a40d0a3df6250cad57c4e/users/7d65ca13f4d44238ae6bfd4ee5786484/roles/176c07b4fe3b4537bdfeb4be21b379c5` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:36.560443 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4d0ed5-9ed0-4b78-a7ab-8ad71aa74a50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8f8077e970a40d0a3df6250cad57c4e, user_id=7d65ca13f4d44238ae6bfd4ee5786484, role_id=176c07b4fe3b4537bdfeb4be21b379c5)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:36.563410 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4d0ed5-9ed0-4b78-a7ab-8ad71aa74a50 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:36.575811 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1160/2314] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:40:36 2022] PUT /identity/v3/projects/a8f8077e970a40d0a3df6250cad57c4e/users/7d65ca13f4d44238ae6bfd4ee5786484/roles/176c07b4fe3b4537bdfeb4be21b379c5 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:40:36.588568 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e5e96a-6abb-4fe9-b232-d61834e46465 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:36.589227 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e5e96a-6abb-4fe9-b232-d61834e46465 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:36.589816 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e5e96a-6abb-4fe9-b232-d61834e46465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:36.621324 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4d0ed5-9ed0-4b78-a7ab-8ad71aa74a50 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:36.622785 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-f1e5e96a-6abb-4fe9-b232-d61834e46465 None None] MFA Rules not processed for user `7d65ca13f4d44238ae6bfd4ee5786484`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:40:36.623064 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-fa4d0ed5-9ed0-4b78-a7ab-8ad71aa74a50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:36.624072 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-f1e5e96a-6abb-4fe9-b232-d61834e46465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:36.704563 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4d0ed5-9ed0-4b78-a7ab-8ad71aa74a50 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:36.709449 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfc6253-accd-4139-8252-dc1d8d3293b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:36.710082 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfc6253-accd-4139-8252-dc1d8d3293b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:36.710363 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfc6253-accd-4139-8252-dc1d8d3293b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:36.710687 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfc6253-accd-4139-8252-dc1d8d3293b9 admin admin] PATH_INFO: `/v3/users/afbb3481efdc4d149cd65de1ba7af484` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:36.711843 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfc6253-accd-4139-8252-dc1d8d3293b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=afbb3481efdc4d149cd65de1ba7af484)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:36.713844 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfc6253-accd-4139-8252-dc1d8d3293b9 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:36.720620 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1155/2315] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:40:36 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 31 12:40:36.736867 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e5e96a-6abb-4fe9-b232-d61834e46465 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:36.740477 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-f1e5e96a-6abb-4fe9-b232-d61834e46465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:36.770093 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:40:36.790597 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-3bfc6253-accd-4139-8252-dc1d8d3293b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afbb3481efdc4d149cd65de1ba7af484'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:36.799483 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-3bfc6253-accd-4139-8252-dc1d8d3293b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afbb3481efdc4d149cd65de1ba7af484'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:36.800537 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-3bfc6253-accd-4139-8252-dc1d8d3293b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afbb3481efdc4d149cd65de1ba7af484'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:36.801175 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-3bfc6253-accd-4139-8252-dc1d8d3293b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afbb3481efdc4d149cd65de1ba7af484'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:36.814555 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-3bfc6253-accd-4139-8252-dc1d8d3293b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afbb3481efdc4d149cd65de1ba7af484'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:36.815801 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=84,keystone:INSERT=6,keystone:DELETE=10 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:40:36.816579 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e5e96a-6abb-4fe9-b232-d61834e46465 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:36.820713 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee20d04b-0c45-4c1b-acdb-0c464b0df970 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d6d7ea077d49e8a66c6c6762499702', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:36.821459 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee20d04b-0c45-4c1b-acdb-0c464b0df970 service cinder] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:36.821739 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee20d04b-0c45-4c1b-acdb-0c464b0df970 service cinder] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:36.822015 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee20d04b-0c45-4c1b-acdb-0c464b0df970 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:36.824060 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-ee20d04b-0c45-4c1b-acdb-0c464b0df970 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:36.829682 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1161/2316] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:40:36 2022] DELETE /identity/v3/users/afbb3481efdc4d149cd65de1ba7af484 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:40:36.835951 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfc6253-accd-4139-8252-dc1d8d3293b9 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:36.837367 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-3bfc6253-accd-4139-8252-dc1d8d3293b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:36.840392 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee20d04b-0c45-4c1b-acdb-0c464b0df970 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:36.841850 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee20d04b-0c45-4c1b-acdb-0c464b0df970 service cinder] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:36.851979 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1156/2317] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:40:36 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:40:36.853172 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5899a03b-ae89-4e5f-b5c5-87d048546961 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:36.853455 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5899a03b-ae89-4e5f-b5c5-87d048546961 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:36.853731 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5899a03b-ae89-4e5f-b5c5-87d048546961 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:36.873590 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-5899a03b-ae89-4e5f-b5c5-87d048546961 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:40:36.874794 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-5899a03b-ae89-4e5f-b5c5-87d048546961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:36.914678 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfc6253-accd-4139-8252-dc1d8d3293b9 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:36.920282 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd06cf3-38f1-4ca7-bd38-3a76d9a9857c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:36.920888 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd06cf3-38f1-4ca7-bd38-3a76d9a9857c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:36.921149 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd06cf3-38f1-4ca7-bd38-3a76d9a9857c admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:36.921452 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd06cf3-38f1-4ca7-bd38-3a76d9a9857c admin admin] PATH_INFO: `/v3/users/7cec7f4586b04d46addee941a03a497c` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:36.922690 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd06cf3-38f1-4ca7-bd38-3a76d9a9857c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cec7f4586b04d46addee941a03a497c)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:36.924982 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd06cf3-38f1-4ca7-bd38-3a76d9a9857c admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:36.954931 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1157/2318] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:40:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:40:36.965286 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5899a03b-ae89-4e5f-b5c5-87d048546961 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:36.966990 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-5899a03b-ae89-4e5f-b5c5-87d048546961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:36.980063 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-ecd06cf3-38f1-4ca7-bd38-3a76d9a9857c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cec7f4586b04d46addee941a03a497c'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:36.988829 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-ecd06cf3-38f1-4ca7-bd38-3a76d9a9857c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cec7f4586b04d46addee941a03a497c'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:36.989470 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-ecd06cf3-38f1-4ca7-bd38-3a76d9a9857c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cec7f4586b04d46addee941a03a497c'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:36.990096 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-ecd06cf3-38f1-4ca7-bd38-3a76d9a9857c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cec7f4586b04d46addee941a03a497c'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:37.003879 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-ecd06cf3-38f1-4ca7-bd38-3a76d9a9857c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cec7f4586b04d46addee941a03a497c'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:37.019819 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1162/2319] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:40:36 2022] DELETE /identity/v3/users/7cec7f4586b04d46addee941a03a497c => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:40:37.059724 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5899a03b-ae89-4e5f-b5c5-87d048546961 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:37.122906 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087d2343-d9b3-47bd-9ff0-ba0821b002c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:37.123562 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087d2343-d9b3-47bd-9ff0-ba0821b002c4 service neutron] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:37.123880 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087d2343-d9b3-47bd-9ff0-ba0821b002c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:37.124118 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087d2343-d9b3-47bd-9ff0-ba0821b002c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:37.126113 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-087d2343-d9b3-47bd-9ff0-ba0821b002c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:37.167810 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:40:37.212021 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087d2343-d9b3-47bd-9ff0-ba0821b002c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:37.214248 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087d2343-d9b3-47bd-9ff0-ba0821b002c4 service neutron] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:37.228135 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd06cf3-38f1-4ca7-bd38-3a76d9a9857c admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:37.229417 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-ecd06cf3-38f1-4ca7-bd38-3a76d9a9857c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:37.244219 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd06cf3-38f1-4ca7-bd38-3a76d9a9857c admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:37.246063 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1158/2320] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:40:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 31 12:40:37.250731 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd23df9-d5aa-43d6-87d5-5129c01cd556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:37.251585 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd23df9-d5aa-43d6-87d5-5129c01cd556 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:37.251938 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd23df9-d5aa-43d6-87d5-5129c01cd556 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:37.252348 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd23df9-d5aa-43d6-87d5-5129c01cd556 admin admin] PATH_INFO: `/v3/projects/437b0ee5c1034635b0c7294f3e1ab629` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:37.253618 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd23df9-d5aa-43d6-87d5-5129c01cd556 admin admin] RBAC: Authorizing `identity:delete_project(project_id=437b0ee5c1034635b0c7294f3e1ab629)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:37.256935 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd23df9-d5aa-43d6-87d5-5129c01cd556 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:37.321169 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-ffd23df9-d5aa-43d6-87d5-5129c01cd556 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '437b0ee5c1034635b0c7294f3e1ab629', 'request_id': 'req-ffd23df9-d5aa-43d6-87d5-5129c01cd556'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:37.328897 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-ffd23df9-d5aa-43d6-87d5-5129c01cd556 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '437b0ee5c1034635b0c7294f3e1ab629', 'request_id': 'req-ffd23df9-d5aa-43d6-87d5-5129c01cd556'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:37.337078 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-ffd23df9-d5aa-43d6-87d5-5129c01cd556 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '437b0ee5c1034635b0c7294f3e1ab629', 'request_id': 'req-ffd23df9-d5aa-43d6-87d5-5129c01cd556'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:37.341133 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:40:37.356474 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-ffd23df9-d5aa-43d6-87d5-5129c01cd556 admin admin] The token cache is being invalidate because project 437b0ee5c1034635b0c7294f3e1ab629 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:40:37.357219 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-ffd23df9-d5aa-43d6-87d5-5129c01cd556 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:37.358306 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-ffd23df9-d5aa-43d6-87d5-5129c01cd556 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:37.359989 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1163/2321] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:40:37 2022] DELETE /identity/v3/projects/437b0ee5c1034635b0c7294f3e1ab629 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 31 12:40:37.366168 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087d2343-d9b3-47bd-9ff0-ba0821b002c4 service neutron] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:37.377595 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087d2343-d9b3-47bd-9ff0-ba0821b002c4 service neutron] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:37.460022 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36db47bf-fde1-4c45-bf30-17d606614dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:37.460656 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36db47bf-fde1-4c45-bf30-17d606614dfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:37.460931 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36db47bf-fde1-4c45-bf30-17d606614dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:37.461201 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36db47bf-fde1-4c45-bf30-17d606614dfe admin admin] PATH_INFO: `/v3/users/7e302c270ece47a895a0fc6d377b929d` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:37.462318 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36db47bf-fde1-4c45-bf30-17d606614dfe admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e302c270ece47a895a0fc6d377b929d)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:37.465231 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36db47bf-fde1-4c45-bf30-17d606614dfe admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:37.522874 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-36db47bf-fde1-4c45-bf30-17d606614dfe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e302c270ece47a895a0fc6d377b929d'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:37.531784 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-36db47bf-fde1-4c45-bf30-17d606614dfe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e302c270ece47a895a0fc6d377b929d'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:37.532415 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-36db47bf-fde1-4c45-bf30-17d606614dfe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e302c270ece47a895a0fc6d377b929d'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:37.549537 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-36db47bf-fde1-4c45-bf30-17d606614dfe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e302c270ece47a895a0fc6d377b929d'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:37.564672 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd23df9-d5aa-43d6-87d5-5129c01cd556 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:37.566342 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-ffd23df9-d5aa-43d6-87d5-5129c01cd556 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:37.578135 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-36db47bf-fde1-4c45-bf30-17d606614dfe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e302c270ece47a895a0fc6d377b929d'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:37.584841 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd23df9-d5aa-43d6-87d5-5129c01cd556 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:37.585959 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1159/2322] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:40:37 2022] DELETE /identity/v3/users/7e302c270ece47a895a0fc6d377b929d => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:40:37.663098 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b4cab4-a736-4db4-b9c4-4dda3a57be25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:37.663784 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b4cab4-a736-4db4-b9c4-4dda3a57be25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:37.664061 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b4cab4-a736-4db4-b9c4-4dda3a57be25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:37.664357 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b4cab4-a736-4db4-b9c4-4dda3a57be25 admin admin] PATH_INFO: `/v3/projects/14a4c39daaf446858c5ae85a5c5666fb` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:37.665556 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b4cab4-a736-4db4-b9c4-4dda3a57be25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=14a4c39daaf446858c5ae85a5c5666fb)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:37.668018 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b4cab4-a736-4db4-b9c4-4dda3a57be25 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:37.728871 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-40b4cab4-a736-4db4-b9c4-4dda3a57be25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14a4c39daaf446858c5ae85a5c5666fb', 'request_id': 'req-40b4cab4-a736-4db4-b9c4-4dda3a57be25'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:37.736214 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-40b4cab4-a736-4db4-b9c4-4dda3a57be25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14a4c39daaf446858c5ae85a5c5666fb', 'request_id': 'req-40b4cab4-a736-4db4-b9c4-4dda3a57be25'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:37.748532 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-40b4cab4-a736-4db4-b9c4-4dda3a57be25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14a4c39daaf446858c5ae85a5c5666fb', 'request_id': 'req-40b4cab4-a736-4db4-b9c4-4dda3a57be25'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:37.763995 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-40b4cab4-a736-4db4-b9c4-4dda3a57be25 admin admin] The token cache is being invalidate because project 14a4c39daaf446858c5ae85a5c5666fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:40:37.764469 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-40b4cab4-a736-4db4-b9c4-4dda3a57be25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:37.765328 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-40b4cab4-a736-4db4-b9c4-4dda3a57be25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:37.770317 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1164/2323] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:40:37 2022] DELETE /identity/v3/projects/14a4c39daaf446858c5ae85a5c5666fb => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 31 12:40:37.771670 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36db47bf-fde1-4c45-bf30-17d606614dfe admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:37.773537 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-36db47bf-fde1-4c45-bf30-17d606614dfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:37.789827 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b4cab4-a736-4db4-b9c4-4dda3a57be25 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:37.791083 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-40b4cab4-a736-4db4-b9c4-4dda3a57be25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:37.796030 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36db47bf-fde1-4c45-bf30-17d606614dfe admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:37.859561 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:40:37.877425 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b4cab4-a736-4db4-b9c4-4dda3a57be25 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:37.880128 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00648db-21d4-4dc0-bec5-73febaf0c7ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:37.880876 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00648db-21d4-4dc0-bec5-73febaf0c7ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:37.881275 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00648db-21d4-4dc0-bec5-73febaf0c7ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:37.882214 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00648db-21d4-4dc0-bec5-73febaf0c7ab admin admin] PATH_INFO: `/v3/users/44e56c28ef7a4b24baae60975d230d30` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:37.882832 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07ba35c-b6da-47b8-829a-df0bec1da81d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:37.883422 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07ba35c-b6da-47b8-829a-df0bec1da81d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:37.883538 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00648db-21d4-4dc0-bec5-73febaf0c7ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=44e56c28ef7a4b24baae60975d230d30)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:37.883821 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07ba35c-b6da-47b8-829a-df0bec1da81d admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:37.883976 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07ba35c-b6da-47b8-829a-df0bec1da81d admin admin] PATH_INFO: `/v3/projects/e2b1110b90b84855b66acd04e23c70de` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:37.885081 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07ba35c-b6da-47b8-829a-df0bec1da81d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2b1110b90b84855b66acd04e23c70de)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:37.885435 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00648db-21d4-4dc0-bec5-73febaf0c7ab admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:37.887487 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07ba35c-b6da-47b8-829a-df0bec1da81d admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:37.915471 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:40:37.946916 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-a07ba35c-b6da-47b8-829a-df0bec1da81d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2b1110b90b84855b66acd04e23c70de', 'request_id': 'req-a07ba35c-b6da-47b8-829a-df0bec1da81d'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:37.953418 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-a07ba35c-b6da-47b8-829a-df0bec1da81d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2b1110b90b84855b66acd04e23c70de', 'request_id': 'req-a07ba35c-b6da-47b8-829a-df0bec1da81d'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:37.960380 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-b00648db-21d4-4dc0-bec5-73febaf0c7ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44e56c28ef7a4b24baae60975d230d30'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:37.961360 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-a07ba35c-b6da-47b8-829a-df0bec1da81d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2b1110b90b84855b66acd04e23c70de', 'request_id': 'req-a07ba35c-b6da-47b8-829a-df0bec1da81d'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:37.971670 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-b00648db-21d4-4dc0-bec5-73febaf0c7ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44e56c28ef7a4b24baae60975d230d30'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:37.972411 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-b00648db-21d4-4dc0-bec5-73febaf0c7ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44e56c28ef7a4b24baae60975d230d30'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:37.978566 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-a07ba35c-b6da-47b8-829a-df0bec1da81d admin admin] The token cache is being invalidate because project e2b1110b90b84855b66acd04e23c70de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:40:37.978914 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-a07ba35c-b6da-47b8-829a-df0bec1da81d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:37.980821 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-a07ba35c-b6da-47b8-829a-df0bec1da81d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:37.982416 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1165/2324] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:40:37 2022] DELETE /identity/v3/projects/e2b1110b90b84855b66acd04e23c70de => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 31 12:40:37.986990 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8202b5-2a44-4ff2-8d0f-aa21009e0513 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:37.987306 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8202b5-2a44-4ff2-8d0f-aa21009e0513 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:37.987584 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8202b5-2a44-4ff2-8d0f-aa21009e0513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:37.988631 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-b00648db-21d4-4dc0-bec5-73febaf0c7ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44e56c28ef7a4b24baae60975d230d30'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:38.001318 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-b00648db-21d4-4dc0-bec5-73febaf0c7ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44e56c28ef7a4b24baae60975d230d30'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:38.008236 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1160/2325] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:40:37 2022] DELETE /identity/v3/users/44e56c28ef7a4b24baae60975d230d30 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 31 12:40:38.009965 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-7e8202b5-2a44-4ff2-8d0f-aa21009e0513 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:40:38.020085 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-7e8202b5-2a44-4ff2-8d0f-aa21009e0513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:38.119944 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1166/2326] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:40:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 31 12:40:38.127796 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2694ae7-6d0d-4b07-9f9b-65715921e592 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:38.128102 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2694ae7-6d0d-4b07-9f9b-65715921e592 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:38.128386 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2694ae7-6d0d-4b07-9f9b-65715921e592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:38.159705 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-f2694ae7-6d0d-4b07-9f9b-65715921e592 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:40:38.161296 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-f2694ae7-6d0d-4b07-9f9b-65715921e592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:38.186831 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1161/2327] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:40:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:40:38.191249 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8202b5-2a44-4ff2-8d0f-aa21009e0513 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:38.202700 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8202b5-2a44-4ff2-8d0f-aa21009e0513 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:38.207648 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a7a77d-4053-417e-a8fc-afe5f3dc0489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:38.208260 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a7a77d-4053-417e-a8fc-afe5f3dc0489 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:38.208588 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a7a77d-4053-417e-a8fc-afe5f3dc0489 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:38.208870 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a7a77d-4053-417e-a8fc-afe5f3dc0489 admin admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:38.209858 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a7a77d-4053-417e-a8fc-afe5f3dc0489 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:40:38.210174 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a7a77d-4053-417e-a8fc-afe5f3dc0489 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:38.212566 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a7a77d-4053-417e-a8fc-afe5f3dc0489 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:38.233170 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1167/2328] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Oct 31 12:40:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:40:38.234265 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2694ae7-6d0d-4b07-9f9b-65715921e592 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:38.243420 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2694ae7-6d0d-4b07-9f9b-65715921e592 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:38.248476 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1f42f1-d576-4513-ad78-cbdba4aec65d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:38.249082 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1f42f1-d576-4513-ad78-cbdba4aec65d admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:38.249356 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1f42f1-d576-4513-ad78-cbdba4aec65d admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:38.249623 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1f42f1-d576-4513-ad78-cbdba4aec65d admin admin] PATH_INFO: `/v3/projects` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:38.250244 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1f42f1-d576-4513-ad78-cbdba4aec65d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:38.252328 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1f42f1-d576-4513-ad78-cbdba4aec65d admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:38.284438 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1162/2329] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:40:38 2022] POST /identity/v3/projects => generated 375 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:40:38.288679 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a7a77d-4053-417e-a8fc-afe5f3dc0489 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:38.300299 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a7a77d-4053-417e-a8fc-afe5f3dc0489 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:38.333706 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1f42f1-d576-4513-ad78-cbdba4aec65d admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:38.335582 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-0f1f42f1-d576-4513-ad78-cbdba4aec65d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:38.375836 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8b115a-9db6-461b-9620-cef0a80605b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:38.376558 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8b115a-9db6-461b-9620-cef0a80605b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:38.376932 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8b115a-9db6-461b-9620-cef0a80605b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:38.377293 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8b115a-9db6-461b-9620-cef0a80605b2 admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:38.378009 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8b115a-9db6-461b-9620-cef0a80605b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:38.380093 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8b115a-9db6-461b-9620-cef0a80605b2 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:38.409242 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1168/2330] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:40:38 2022] POST /identity/v3/users => generated 288 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 31 12:40:38.410173 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8b115a-9db6-461b-9620-cef0a80605b2 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:38.424349 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8b115a-9db6-461b-9620-cef0a80605b2 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:38.428048 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1f42f1-d576-4513-ad78-cbdba4aec65d admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:38.428946 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ab9a1b-b4f9-4d69-8977-311d1c37fe84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:38.429542 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ab9a1b-b4f9-4d69-8977-311d1c37fe84 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:38.430056 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ab9a1b-b4f9-4d69-8977-311d1c37fe84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:38.430330 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ab9a1b-b4f9-4d69-8977-311d1c37fe84 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:38.430966 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ab9a1b-b4f9-4d69-8977-311d1c37fe84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:38.433151 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ab9a1b-b4f9-4d69-8977-311d1c37fe84 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:38.434111 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230e20c1-2752-48c4-b09f-aacd4235cf34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:38.434959 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230e20c1-2752-48c4-b09f-aacd4235cf34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:38.435448 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230e20c1-2752-48c4-b09f-aacd4235cf34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:38.435808 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230e20c1-2752-48c4-b09f-aacd4235cf34 admin admin] PATH_INFO: `/v3/projects/18862a8e83f54cc793c90928b7af39ce` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:38.437149 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230e20c1-2752-48c4-b09f-aacd4235cf34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=18862a8e83f54cc793c90928b7af39ce)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:38.439113 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230e20c1-2752-48c4-b09f-aacd4235cf34 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:38.444461 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1169/2331] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:40:38 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:40:38.449699 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ab9a1b-b4f9-4d69-8977-311d1c37fe84 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:38.460375 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ab9a1b-b4f9-4d69-8977-311d1c37fe84 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:38.469217 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e77b419-23da-493b-9a6f-e4245d5c6de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:38.469899 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e77b419-23da-493b-9a6f-e4245d5c6de4 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:38.470184 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e77b419-23da-493b-9a6f-e4245d5c6de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:38.470457 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e77b419-23da-493b-9a6f-e4245d5c6de4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:38.471483 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e77b419-23da-493b-9a6f-e4245d5c6de4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:38.473701 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e77b419-23da-493b-9a6f-e4245d5c6de4 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:38.487146 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1170/2332] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:40:38 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 31 12:40:38.491839 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e77b419-23da-493b-9a6f-e4245d5c6de4 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:38.493164 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:40:38.506018 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e77b419-23da-493b-9a6f-e4245d5c6de4 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:38.506390 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-230e20c1-2752-48c4-b09f-aacd4235cf34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18862a8e83f54cc793c90928b7af39ce', 'request_id': 'req-230e20c1-2752-48c4-b09f-aacd4235cf34'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:38.509889 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:40:38.512663 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-230e20c1-2752-48c4-b09f-aacd4235cf34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18862a8e83f54cc793c90928b7af39ce', 'request_id': 'req-230e20c1-2752-48c4-b09f-aacd4235cf34'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:38.521741 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-230e20c1-2752-48c4-b09f-aacd4235cf34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18862a8e83f54cc793c90928b7af39ce', 'request_id': 'req-230e20c1-2752-48c4-b09f-aacd4235cf34'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:38.533448 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-230e20c1-2752-48c4-b09f-aacd4235cf34 admin admin] The token cache is being invalidate because project 18862a8e83f54cc793c90928b7af39ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:40:38.533729 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-230e20c1-2752-48c4-b09f-aacd4235cf34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:38.534367 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-230e20c1-2752-48c4-b09f-aacd4235cf34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:38.536568 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1163/2333] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:40:38 2022] DELETE /identity/v3/projects/18862a8e83f54cc793c90928b7af39ce => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 31 12:40:38.537315 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230e20c1-2752-48c4-b09f-aacd4235cf34 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:38.538600 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-230e20c1-2752-48c4-b09f-aacd4235cf34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:38.571181 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b256e05b-7f08-4686-a545-c6f66a1d142a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:38.571860 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b256e05b-7f08-4686-a545-c6f66a1d142a admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:38.572137 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b256e05b-7f08-4686-a545-c6f66a1d142a admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:38.572403 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b256e05b-7f08-4686-a545-c6f66a1d142a admin admin] PATH_INFO: `/v3/projects/2a757c55f81c4d14a9e17a67a2f17d56/users/a70671eaf2e44a9d8f685191948da65f/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:38.586716 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b256e05b-7f08-4686-a545-c6f66a1d142a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a757c55f81c4d14a9e17a67a2f17d56, user_id=a70671eaf2e44a9d8f685191948da65f, role_id=ca1a5695f8bd499aa0b8cedaeecd2cc6)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:38.589889 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b256e05b-7f08-4686-a545-c6f66a1d142a admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:38.602590 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1171/2334] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:40:38 2022] PUT /identity/v3/projects/2a757c55f81c4d14a9e17a67a2f17d56/users/a70671eaf2e44a9d8f685191948da65f/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 31 12:40:38.610271 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230e20c1-2752-48c4-b09f-aacd4235cf34 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:38.610381 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d735729d-50cc-4497-b5e7-6af328f5fbd3 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:38.610381 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d735729d-50cc-4497-b5e7-6af328f5fbd3 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:38.610381 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d735729d-50cc-4497-b5e7-6af328f5fbd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:38.627253 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-d735729d-50cc-4497-b5e7-6af328f5fbd3 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:40:38.628459 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-d735729d-50cc-4497-b5e7-6af328f5fbd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:38.664792 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220b2b56-6677-4615-ad0c-de05caf64df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:38.665424 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220b2b56-6677-4615-ad0c-de05caf64df5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:38.665705 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220b2b56-6677-4615-ad0c-de05caf64df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:38.665977 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220b2b56-6677-4615-ad0c-de05caf64df5 admin admin] PATH_INFO: `/v3/users/9f58c4ba32254b77b33c6bcc26d8852c` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:38.667152 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220b2b56-6677-4615-ad0c-de05caf64df5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f58c4ba32254b77b33c6bcc26d8852c)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:38.669508 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220b2b56-6677-4615-ad0c-de05caf64df5 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:38.691273 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1172/2335] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:40:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:40:38.692612 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eac752-afb5-46d0-bc35-ebcdd383da59 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:38.692900 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eac752-afb5-46d0-bc35-ebcdd383da59 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:38.693293 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eac752-afb5-46d0-bc35-ebcdd383da59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:38.722670 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-28eac752-afb5-46d0-bc35-ebcdd383da59 None None] MFA Rules not processed for user `a70671eaf2e44a9d8f685191948da65f`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:40:38.723585 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-220b2b56-6677-4615-ad0c-de05caf64df5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f58c4ba32254b77b33c6bcc26d8852c'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:38.723925 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-28eac752-afb5-46d0-bc35-ebcdd383da59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:38.731137 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-220b2b56-6677-4615-ad0c-de05caf64df5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f58c4ba32254b77b33c6bcc26d8852c'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:38.731853 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-220b2b56-6677-4615-ad0c-de05caf64df5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f58c4ba32254b77b33c6bcc26d8852c'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:38.743949 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-220b2b56-6677-4615-ad0c-de05caf64df5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f58c4ba32254b77b33c6bcc26d8852c'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:38.746874 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:40:38.754944 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-220b2b56-6677-4615-ad0c-de05caf64df5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f58c4ba32254b77b33c6bcc26d8852c'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:38.761503 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1164/2336] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:40:38 2022] DELETE /identity/v3/users/9f58c4ba32254b77b33c6bcc26d8852c => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:40:38.762700 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a8ef0a-46a9-4f3b-899b-9b85e0513493 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:38.762986 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a8ef0a-46a9-4f3b-899b-9b85e0513493 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:38.763392 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a8ef0a-46a9-4f3b-899b-9b85e0513493 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:38.783688 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-27a8ef0a-46a9-4f3b-899b-9b85e0513493 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:40:38.784877 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-27a8ef0a-46a9-4f3b-899b-9b85e0513493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:38.798482 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1173/2337] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:40:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:40:38.799969 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ec1a6b-f4c0-4f79-85c2-38932ff74d21 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:38.800247 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ec1a6b-f4c0-4f79-85c2-38932ff74d21 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:38.800528 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ec1a6b-f4c0-4f79-85c2-38932ff74d21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:38.820992 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-03ec1a6b-f4c0-4f79-85c2-38932ff74d21 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:40:38.822183 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-03ec1a6b-f4c0-4f79-85c2-38932ff74d21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:38.835474 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:40:38.850685 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1165/2338] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:40:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:40:38.851548 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a8ef0a-46a9-4f3b-899b-9b85e0513493 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:38.852787 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-27a8ef0a-46a9-4f3b-899b-9b85e0513493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:38.884119 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1174/2339] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:40:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:40:38.885043 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ec1a6b-f4c0-4f79-85c2-38932ff74d21 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:38.896292 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ec1a6b-f4c0-4f79-85c2-38932ff74d21 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:38.900989 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e717309-1129-4f6a-b877-c3780c5f0fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:38.901778 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e717309-1129-4f6a-b877-c3780c5f0fd5 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:38.902055 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e717309-1129-4f6a-b877-c3780c5f0fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:38.902323 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e717309-1129-4f6a-b877-c3780c5f0fd5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:38.902959 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e717309-1129-4f6a-b877-c3780c5f0fd5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:40:38.903293 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e717309-1129-4f6a-b877-c3780c5f0fd5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:38.904869 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e717309-1129-4f6a-b877-c3780c5f0fd5 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:38.918269 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1175/2340] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Oct 31 12:40:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:40:38.919200 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e717309-1129-4f6a-b877-c3780c5f0fd5 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:38.920851 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-1e717309-1129-4f6a-b877-c3780c5f0fd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:38.922215 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a8ef0a-46a9-4f3b-899b-9b85e0513493 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:38.926043 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d641e6-495a-4e5c-816e-89734146eb2e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d6d7ea077d49e8a66c6c6762499702', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:38.926719 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d641e6-495a-4e5c-816e-89734146eb2e service cinder] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:38.927070 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d641e6-495a-4e5c-816e-89734146eb2e service cinder] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:38.927450 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d641e6-495a-4e5c-816e-89734146eb2e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:38.977896 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e717309-1129-4f6a-b877-c3780c5f0fd5 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:38.982678 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b49c406-62bd-4b7c-9754-a67ae7d9effe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:38.983320 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b49c406-62bd-4b7c-9754-a67ae7d9effe service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:38.983600 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b49c406-62bd-4b7c-9754-a67ae7d9effe service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:38.983873 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b49c406-62bd-4b7c-9754-a67ae7d9effe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:38.987365 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d641e6-495a-4e5c-816e-89734146eb2e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:38.988841 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d641e6-495a-4e5c-816e-89734146eb2e service cinder] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:38.992924 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b49c406-62bd-4b7c-9754-a67ae7d9effe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:38.994485 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b49c406-62bd-4b7c-9754-a67ae7d9effe service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:38.999508 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1166/2341] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:40:38 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:40:39.000288 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d641e6-495a-4e5c-816e-89734146eb2e service cinder] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:39.004604 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1176/2342] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:40:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:40:39.013779 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d641e6-495a-4e5c-816e-89734146eb2e service cinder] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:39.018842 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a49a6d-66fe-4406-9cb9-085f22d4677f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:39.019409 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a49a6d-66fe-4406-9cb9-085f22d4677f admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:39.019688 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a49a6d-66fe-4406-9cb9-085f22d4677f admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:39.019958 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a49a6d-66fe-4406-9cb9-085f22d4677f admin admin] PATH_INFO: `/v3/projects` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:39.020764 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a49a6d-66fe-4406-9cb9-085f22d4677f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:39.022657 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a49a6d-66fe-4406-9cb9-085f22d4677f admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:39.054359 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1167/2343] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:40:38 2022] POST /identity/v3/projects => generated 357 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:40:39.059068 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b49c406-62bd-4b7c-9754-a67ae7d9effe service neutron] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:39.070691 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b49c406-62bd-4b7c-9754-a67ae7d9effe service neutron] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:39.138582 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c7d9f9-57db-4357-b6db-0ab92447357e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:39.139230 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c7d9f9-57db-4357-b6db-0ab92447357e admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:39.139515 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c7d9f9-57db-4357-b6db-0ab92447357e admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:39.140259 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c7d9f9-57db-4357-b6db-0ab92447357e admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:39.140396 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c7d9f9-57db-4357-b6db-0ab92447357e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:39.142283 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c7d9f9-57db-4357-b6db-0ab92447357e admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:39.164776 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1177/2344] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:40:39 2022] POST /identity/v3/users => generated 279 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:40:39.169282 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a49a6d-66fe-4406-9cb9-085f22d4677f admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:39.192153 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a49a6d-66fe-4406-9cb9-085f22d4677f admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:39.198585 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a90b79-bb06-4724-8df3-ccf0418ca22f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:39.198737 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a90b79-bb06-4724-8df3-ccf0418ca22f admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:39.199024 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a90b79-bb06-4724-8df3-ccf0418ca22f admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:39.199341 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a90b79-bb06-4724-8df3-ccf0418ca22f admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:39.199991 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a90b79-bb06-4724-8df3-ccf0418ca22f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:39.201734 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a90b79-bb06-4724-8df3-ccf0418ca22f admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:39.217524 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1168/2345] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:40:39 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:40:39.219466 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c7d9f9-57db-4357-b6db-0ab92447357e admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:39.230830 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c7d9f9-57db-4357-b6db-0ab92447357e admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:39.236432 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ed2f20-5484-4178-8ab8-4d19ca679e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:39.236998 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ed2f20-5484-4178-8ab8-4d19ca679e64 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:39.237310 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ed2f20-5484-4178-8ab8-4d19ca679e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:39.237584 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ed2f20-5484-4178-8ab8-4d19ca679e64 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:39.238230 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ed2f20-5484-4178-8ab8-4d19ca679e64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:39.239850 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ed2f20-5484-4178-8ab8-4d19ca679e64 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:39.255946 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1178/2346] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:40:39 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:40:39.257350 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a90b79-bb06-4724-8df3-ccf0418ca22f admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:39.268965 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a90b79-bb06-4724-8df3-ccf0418ca22f admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:39.274275 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccd7da5-f247-43be-975c-681d0233bd9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:39.275041 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccd7da5-f247-43be-975c-681d0233bd9b admin admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:39.284186 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccd7da5-f247-43be-975c-681d0233bd9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:39.285117 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccd7da5-f247-43be-975c-681d0233bd9b admin admin] PATH_INFO: `/v3/projects/ab689a8695914fe3a9004e6603eb3c0f/users/c4b03b84194642e8a0d27ce0edb7a58b/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:39.320845 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:40:39.322582 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ed2f20-5484-4178-8ab8-4d19ca679e64 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:39.325153 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-e3ed2f20-5484-4178-8ab8-4d19ca679e64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:39.329282 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccd7da5-f247-43be-975c-681d0233bd9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab689a8695914fe3a9004e6603eb3c0f, user_id=c4b03b84194642e8a0d27ce0edb7a58b, role_id=ca1a5695f8bd499aa0b8cedaeecd2cc6)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:39.343300 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ed2f20-5484-4178-8ab8-4d19ca679e64 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:39.345270 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccd7da5-f247-43be-975c-681d0233bd9b admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:39.349590 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09bb5f1-bebc-4697-8a5f-548e771abe55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:39.350457 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09bb5f1-bebc-4697-8a5f-548e771abe55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:39.351334 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09bb5f1-bebc-4697-8a5f-548e771abe55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:39.351805 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09bb5f1-bebc-4697-8a5f-548e771abe55 admin admin] PATH_INFO: `/v3/projects/8e32724626464d01bd7f866693de0a42` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:39.353391 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09bb5f1-bebc-4697-8a5f-548e771abe55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e32724626464d01bd7f866693de0a42)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:39.356472 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09bb5f1-bebc-4697-8a5f-548e771abe55 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:39.382292 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1169/2347] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:40:39 2022] PUT /identity/v3/projects/ab689a8695914fe3a9004e6603eb3c0f/users/c4b03b84194642e8a0d27ce0edb7a58b/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:40:39.383842 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eedd756-1133-4e1a-b797-740609d1b0fe None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:39.385107 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eedd756-1133-4e1a-b797-740609d1b0fe None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:39.385567 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eedd756-1133-4e1a-b797-740609d1b0fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:39.391790 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:40:39.425068 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-3eedd756-1133-4e1a-b797-740609d1b0fe None None] MFA Rules not processed for user `c4b03b84194642e8a0d27ce0edb7a58b`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:40:39.426538 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-3eedd756-1133-4e1a-b797-740609d1b0fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:39.434415 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-e09bb5f1-bebc-4697-8a5f-548e771abe55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e32724626464d01bd7f866693de0a42', 'request_id': 'req-e09bb5f1-bebc-4697-8a5f-548e771abe55'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:39.441675 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-e09bb5f1-bebc-4697-8a5f-548e771abe55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e32724626464d01bd7f866693de0a42', 'request_id': 'req-e09bb5f1-bebc-4697-8a5f-548e771abe55'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:39.450384 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-e09bb5f1-bebc-4697-8a5f-548e771abe55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e32724626464d01bd7f866693de0a42', 'request_id': 'req-e09bb5f1-bebc-4697-8a5f-548e771abe55'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:39.460919 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-e09bb5f1-bebc-4697-8a5f-548e771abe55 admin admin] The token cache is being invalidate because project 8e32724626464d01bd7f866693de0a42 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:40:39.461212 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-e09bb5f1-bebc-4697-8a5f-548e771abe55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:39.461862 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-e09bb5f1-bebc-4697-8a5f-548e771abe55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:39.463396 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1179/2348] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:40:39 2022] DELETE /identity/v3/projects/8e32724626464d01bd7f866693de0a42 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:40:39.467741 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09bb5f1-bebc-4697-8a5f-548e771abe55 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:39.469075 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-e09bb5f1-bebc-4697-8a5f-548e771abe55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:39.497293 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1170/2349] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:40:39 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 31 12:40:39.501692 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eedd756-1133-4e1a-b797-740609d1b0fe None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:39.503039 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-3eedd756-1133-4e1a-b797-740609d1b0fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:39.538533 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09bb5f1-bebc-4697-8a5f-548e771abe55 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:39.543601 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f33e42c-2db8-4174-8d53-4828d7a16ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:39.544235 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f33e42c-2db8-4174-8d53-4828d7a16ed8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:39.544530 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f33e42c-2db8-4174-8d53-4828d7a16ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:39.544803 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f33e42c-2db8-4174-8d53-4828d7a16ed8 admin admin] PATH_INFO: `/v3/users/7d65ca13f4d44238ae6bfd4ee5786484` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:39.545957 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f33e42c-2db8-4174-8d53-4828d7a16ed8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d65ca13f4d44238ae6bfd4ee5786484)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:39.547979 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f33e42c-2db8-4174-8d53-4828d7a16ed8 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:39.571319 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eedd756-1133-4e1a-b797-740609d1b0fe None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:39.577216 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088cd08a-a093-4350-a7e5-7cde5dc748f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:39.577853 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088cd08a-a093-4350-a7e5-7cde5dc748f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:39.578225 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088cd08a-a093-4350-a7e5-7cde5dc748f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:39.578648 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088cd08a-a093-4350-a7e5-7cde5dc748f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:39.579496 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088cd08a-a093-4350-a7e5-7cde5dc748f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:39.582034 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088cd08a-a093-4350-a7e5-7cde5dc748f2 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:39.598120 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-5f33e42c-2db8-4174-8d53-4828d7a16ed8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d65ca13f4d44238ae6bfd4ee5786484'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:39.605678 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-5f33e42c-2db8-4174-8d53-4828d7a16ed8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d65ca13f4d44238ae6bfd4ee5786484'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:39.606494 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-5f33e42c-2db8-4174-8d53-4828d7a16ed8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d65ca13f4d44238ae6bfd4ee5786484'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:39.607113 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-5f33e42c-2db8-4174-8d53-4828d7a16ed8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d65ca13f4d44238ae6bfd4ee5786484'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:39.611563 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1171/2350] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:40:39 2022] POST /identity/v3/projects => generated 355 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 31 12:40:39.615615 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088cd08a-a093-4350-a7e5-7cde5dc748f2 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:39.617309 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-088cd08a-a093-4350-a7e5-7cde5dc748f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:39.620007 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-5f33e42c-2db8-4174-8d53-4828d7a16ed8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d65ca13f4d44238ae6bfd4ee5786484'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:39.633166 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1180/2351] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:40:39 2022] DELETE /identity/v3/users/7d65ca13f4d44238ae6bfd4ee5786484 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 31 12:40:39.685275 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088cd08a-a093-4350-a7e5-7cde5dc748f2 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:39.693909 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:40:39.747488 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bf0625-ee82-4681-bdaf-f0becc0b0016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:39.748102 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bf0625-ee82-4681-bdaf-f0becc0b0016 admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:39.748380 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bf0625-ee82-4681-bdaf-f0becc0b0016 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:39.748651 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bf0625-ee82-4681-bdaf-f0becc0b0016 admin admin] PATH_INFO: `/v3/users` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:39.749285 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bf0625-ee82-4681-bdaf-f0becc0b0016 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:39.751217 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bf0625-ee82-4681-bdaf-f0becc0b0016 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:39.776376 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1172/2352] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:40:39 2022] POST /identity/v3/users => generated 284 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 31 12:40:39.777740 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f33e42c-2db8-4174-8d53-4828d7a16ed8 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:39.788879 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f33e42c-2db8-4174-8d53-4828d7a16ed8 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:39.797135 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2b28fb-171b-449b-be1a-769d6f349cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:39.798001 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2b28fb-171b-449b-be1a-769d6f349cf5 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:39.798338 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2b28fb-171b-449b-be1a-769d6f349cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:39.798695 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2b28fb-171b-449b-be1a-769d6f349cf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:39.799394 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2b28fb-171b-449b-be1a-769d6f349cf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:39.801034 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2b28fb-171b-449b-be1a-769d6f349cf5 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:39.812545 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1181/2353] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:40:39 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:40:39.816763 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bf0625-ee82-4681-bdaf-f0becc0b0016 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:39.826851 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bf0625-ee82-4681-bdaf-f0becc0b0016 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:39.831785 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe456cb0-acd8-453c-be16-13e9a6156b61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:39.832440 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe456cb0-acd8-453c-be16-13e9a6156b61 admin admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:39.832720 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe456cb0-acd8-453c-be16-13e9a6156b61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:39.832992 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe456cb0-acd8-453c-be16-13e9a6156b61 admin admin] PATH_INFO: `/v3/projects/710e09d9dd3044b9b65d214269cbc1e5/users/2fada062ec9a405986a35df1e5d6d356/roles/176c07b4fe3b4537bdfeb4be21b379c5` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:39.849288 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe456cb0-acd8-453c-be16-13e9a6156b61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=710e09d9dd3044b9b65d214269cbc1e5, user_id=2fada062ec9a405986a35df1e5d6d356, role_id=176c07b4fe3b4537bdfeb4be21b379c5)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:39.852070 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe456cb0-acd8-453c-be16-13e9a6156b61 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:39.866188 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1173/2354] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:40:39 2022] PUT /identity/v3/projects/710e09d9dd3044b9b65d214269cbc1e5/users/2fada062ec9a405986a35df1e5d6d356/roles/176c07b4fe3b4537bdfeb4be21b379c5 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:40:39.871070 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2b28fb-171b-449b-be1a-769d6f349cf5 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:39.880969 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2b28fb-171b-449b-be1a-769d6f349cf5 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:39.898495 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:40:39.928985 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe456cb0-acd8-453c-be16-13e9a6156b61 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:39.931068 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-fe456cb0-acd8-453c-be16-13e9a6156b61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:39.946181 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5038e665-771b-4b7b-bc46-12b8835b7560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:39.946942 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5038e665-771b-4b7b-bc46-12b8835b7560 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:39.947264 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5038e665-771b-4b7b-bc46-12b8835b7560 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:39.947540 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5038e665-771b-4b7b-bc46-12b8835b7560 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:39.948257 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5038e665-771b-4b7b-bc46-12b8835b7560 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:39.950650 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5038e665-771b-4b7b-bc46-12b8835b7560 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:39.967012 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1182/2355] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:40:39 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 31 12:40:39.970902 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5038e665-771b-4b7b-bc46-12b8835b7560 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:39.981943 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5038e665-771b-4b7b-bc46-12b8835b7560 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:39.986685 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdfcb01-ca37-40d2-a6a8-05dd2ab5de5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:39.987386 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdfcb01-ca37-40d2-a6a8-05dd2ab5de5b admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:39.987669 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdfcb01-ca37-40d2-a6a8-05dd2ab5de5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:39.987945 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdfcb01-ca37-40d2-a6a8-05dd2ab5de5b admin admin] PATH_INFO: `/v3/projects/710e09d9dd3044b9b65d214269cbc1e5/users/2fada062ec9a405986a35df1e5d6d356/roles/176c07b4fe3b4537bdfeb4be21b379c5` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:39.989857 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdfcb01-ca37-40d2-a6a8-05dd2ab5de5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=710e09d9dd3044b9b65d214269cbc1e5, user_id=2fada062ec9a405986a35df1e5d6d356, role_id=176c07b4fe3b4537bdfeb4be21b379c5)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:39.992485 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdfcb01-ca37-40d2-a6a8-05dd2ab5de5b admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:39.999846 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe456cb0-acd8-453c-be16-13e9a6156b61 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:40.004539 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1183/2356] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:40:39 2022] PUT /identity/v3/projects/710e09d9dd3044b9b65d214269cbc1e5/users/2fada062ec9a405986a35df1e5d6d356/roles/176c07b4fe3b4537bdfeb4be21b379c5 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:40:40.015949 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82207cdb-3afe-441d-a529-2f7f6207d57d None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:40.016231 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82207cdb-3afe-441d-a529-2f7f6207d57d None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:40.016504 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82207cdb-3afe-441d-a529-2f7f6207d57d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:40.045931 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-82207cdb-3afe-441d-a529-2f7f6207d57d None None] MFA Rules not processed for user `2fada062ec9a405986a35df1e5d6d356`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:40:40.047183 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-82207cdb-3afe-441d-a529-2f7f6207d57d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:40.062139 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6cdec9-1321-49fb-b3b4-c251286b327e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:40.062776 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6cdec9-1321-49fb-b3b4-c251286b327e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:40.063055 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6cdec9-1321-49fb-b3b4-c251286b327e admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:40.063357 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6cdec9-1321-49fb-b3b4-c251286b327e admin admin] PATH_INFO: `/v3/projects/a8f8077e970a40d0a3df6250cad57c4e` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:40.064483 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6cdec9-1321-49fb-b3b4-c251286b327e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8f8077e970a40d0a3df6250cad57c4e)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:40.066374 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6cdec9-1321-49fb-b3b4-c251286b327e admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:40.097976 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:40:40.116516 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-eb6cdec9-1321-49fb-b3b4-c251286b327e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8f8077e970a40d0a3df6250cad57c4e', 'request_id': 'req-eb6cdec9-1321-49fb-b3b4-c251286b327e'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:40.117693 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1184/2357] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:40:40 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 31 12:40:40.123085 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-eb6cdec9-1321-49fb-b3b4-c251286b327e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8f8077e970a40d0a3df6250cad57c4e', 'request_id': 'req-eb6cdec9-1321-49fb-b3b4-c251286b327e'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:40.130009 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82207cdb-3afe-441d-a529-2f7f6207d57d None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:40.130674 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-eb6cdec9-1321-49fb-b3b4-c251286b327e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8f8077e970a40d0a3df6250cad57c4e', 'request_id': 'req-eb6cdec9-1321-49fb-b3b4-c251286b327e'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:40.131825 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-82207cdb-3afe-441d-a529-2f7f6207d57d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:40.142495 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-eb6cdec9-1321-49fb-b3b4-c251286b327e admin admin] The token cache is being invalidate because project a8f8077e970a40d0a3df6250cad57c4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:40:40.143035 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-eb6cdec9-1321-49fb-b3b4-c251286b327e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:40.143720 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-eb6cdec9-1321-49fb-b3b4-c251286b327e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:40.145214 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1174/2358] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:40:39 2022] DELETE /identity/v3/projects/a8f8077e970a40d0a3df6250cad57c4e => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 31 12:40:40.149819 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3aa1b5-8446-4b91-95fc-4281f2e6543e None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:40.150107 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3aa1b5-8446-4b91-95fc-4281f2e6543e None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:40.150382 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3aa1b5-8446-4b91-95fc-4281f2e6543e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:40.171392 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-ed3aa1b5-8446-4b91-95fc-4281f2e6543e None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:40:40.172606 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-ed3aa1b5-8446-4b91-95fc-4281f2e6543e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:40.192598 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82207cdb-3afe-441d-a529-2f7f6207d57d None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:40.232888 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1175/2359] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:40:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 31 12:40:40.234970 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:40:40.241324 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9dcfd4-f36a-421a-bba6-c6bb372b0caa None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:40.241607 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9dcfd4-f36a-421a-bba6-c6bb372b0caa None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:40.242009 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9dcfd4-f36a-421a-bba6-c6bb372b0caa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:40.243792 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c5aa73-e638-4749-8c65-7a40f6ec3599 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d6d7ea077d49e8a66c6c6762499702', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:40.244406 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c5aa73-e638-4749-8c65-7a40f6ec3599 service cinder] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:40.244684 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c5aa73-e638-4749-8c65-7a40f6ec3599 service cinder] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:40.245504 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c5aa73-e638-4749-8c65-7a40f6ec3599 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:40.247610 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-11c5aa73-e638-4749-8c65-7a40f6ec3599 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:40.262296 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-2f9dcfd4-f36a-421a-bba6-c6bb372b0caa None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:40:40.263496 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-2f9dcfd4-f36a-421a-bba6-c6bb372b0caa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:40.271880 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1176/2360] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:40:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 31 12:40:40.275868 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9dcfd4-f36a-421a-bba6-c6bb372b0caa None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:40.285399 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9dcfd4-f36a-421a-bba6-c6bb372b0caa None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:40.290096 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2090db6b-73fd-4417-81bb-f93d1578117d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:40.290676 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2090db6b-73fd-4417-81bb-f93d1578117d admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:40.290961 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2090db6b-73fd-4417-81bb-f93d1578117d admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:40.291269 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2090db6b-73fd-4417-81bb-f93d1578117d admin admin] PATH_INFO: `/v3/domains` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:40.291985 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2090db6b-73fd-4417-81bb-f93d1578117d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:40:40.292280 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2090db6b-73fd-4417-81bb-f93d1578117d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:40.293901 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2090db6b-73fd-4417-81bb-f93d1578117d admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:40.305398 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1177/2361] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Oct 31 12:40:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 31 12:40:40.310280 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2090db6b-73fd-4417-81bb-f93d1578117d admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:40.312696 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c5aa73-e638-4749-8c65-7a40f6ec3599 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:40.314111 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c5aa73-e638-4749-8c65-7a40f6ec3599 service cinder] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:40.319732 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2090db6b-73fd-4417-81bb-f93d1578117d admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:40.324715 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5efbfa-5c45-4a95-bf9c-f9590afd965b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:40.324881 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1185/2362] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:40:40 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 31 12:40:40.325296 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5efbfa-5c45-4a95-bf9c-f9590afd965b admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:40.325567 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5efbfa-5c45-4a95-bf9c-f9590afd965b admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:40.325832 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5efbfa-5c45-4a95-bf9c-f9590afd965b admin admin] PATH_INFO: `/v3/projects` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:40.326449 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5efbfa-5c45-4a95-bf9c-f9590afd965b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:40.328479 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5efbfa-5c45-4a95-bf9c-f9590afd965b admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:40.357095 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1178/2363] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:40:40 2022] POST /identity/v3/projects => generated 389 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 31 12:40:40.361065 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c5aa73-e638-4749-8c65-7a40f6ec3599 service cinder] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:40.371566 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c5aa73-e638-4749-8c65-7a40f6ec3599 service cinder] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:40.401949 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5efbfa-5c45-4a95-bf9c-f9590afd965b admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:40.403753 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-ea5efbfa-5c45-4a95-bf9c-f9590afd965b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:40.441645 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6da4238-1b9a-4da4-9ced-3ff74c5df20f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:40.442269 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6da4238-1b9a-4da4-9ced-3ff74c5df20f admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:40.442550 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6da4238-1b9a-4da4-9ced-3ff74c5df20f admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:40.442828 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6da4238-1b9a-4da4-9ced-3ff74c5df20f admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:40.443634 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6da4238-1b9a-4da4-9ced-3ff74c5df20f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:40.445596 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6da4238-1b9a-4da4-9ced-3ff74c5df20f admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:40.469162 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5efbfa-5c45-4a95-bf9c-f9590afd965b admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:40.471239 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1186/2364] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:40:40 2022] POST /identity/v3/users => generated 295 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:40:40.474181 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6da4238-1b9a-4da4-9ced-3ff74c5df20f admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:40.475353 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cdb9c6-74d5-4da3-b8c0-a8e37c9930e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d6d7ea077d49e8a66c6c6762499702', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:40.475998 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cdb9c6-74d5-4da3-b8c0-a8e37c9930e1 service cinder] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:40.476286 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cdb9c6-74d5-4da3-b8c0-a8e37c9930e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:40.476557 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cdb9c6-74d5-4da3-b8c0-a8e37c9930e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:40.478303 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-d1cdb9c6-74d5-4da3-b8c0-a8e37c9930e1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:40.485441 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6da4238-1b9a-4da4-9ced-3ff74c5df20f admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:40.490366 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ce515a-0ec0-4fa2-b2b4-47b70eef3e15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:40.491122 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ce515a-0ec0-4fa2-b2b4-47b70eef3e15 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:40.491436 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ce515a-0ec0-4fa2-b2b4-47b70eef3e15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:40.491704 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ce515a-0ec0-4fa2-b2b4-47b70eef3e15 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:40.492346 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ce515a-0ec0-4fa2-b2b4-47b70eef3e15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:40.493963 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ce515a-0ec0-4fa2-b2b4-47b70eef3e15 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:40.504946 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1187/2365] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:40:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 31 12:40:40.508795 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ce515a-0ec0-4fa2-b2b4-47b70eef3e15 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:40.516369 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:40:40.520456 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ce515a-0ec0-4fa2-b2b4-47b70eef3e15 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:40.527263 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f99a53-3907-4368-bcb4-12a159d99f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:40.527564 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f99a53-3907-4368-bcb4-12a159d99f02 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:40.527845 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f99a53-3907-4368-bcb4-12a159d99f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:40.528114 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f99a53-3907-4368-bcb4-12a159d99f02 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:40.528927 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f99a53-3907-4368-bcb4-12a159d99f02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:40.530556 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f99a53-3907-4368-bcb4-12a159d99f02 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:40.543394 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1188/2366] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:40:40 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 31 12:40:40.547401 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f99a53-3907-4368-bcb4-12a159d99f02 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:40.559401 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f99a53-3907-4368-bcb4-12a159d99f02 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:40.562043 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cdb9c6-74d5-4da3-b8c0-a8e37c9930e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:40.563848 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cdb9c6-74d5-4da3-b8c0-a8e37c9930e1 service cinder] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:40.565406 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e649973d-2143-4f37-912e-ff6ebdfb871e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:40.566329 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e649973d-2143-4f37-912e-ff6ebdfb871e admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:40.566592 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e649973d-2143-4f37-912e-ff6ebdfb871e admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:40.566937 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e649973d-2143-4f37-912e-ff6ebdfb871e admin admin] PATH_INFO: `/v3/projects/e5495f09e3804dcda8b5a2cdb2c056aa/users/12f4cebe3aa74c718651a34613244dac/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:40.579810 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1179/2367] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:40:40 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 31 12:40:40.584894 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e649973d-2143-4f37-912e-ff6ebdfb871e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5495f09e3804dcda8b5a2cdb2c056aa, user_id=12f4cebe3aa74c718651a34613244dac, role_id=ca1a5695f8bd499aa0b8cedaeecd2cc6)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:40.587801 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e649973d-2143-4f37-912e-ff6ebdfb871e admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:40.612624 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1189/2368] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:40:40 2022] PUT /identity/v3/projects/e5495f09e3804dcda8b5a2cdb2c056aa/users/12f4cebe3aa74c718651a34613244dac/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 31 12:40:40.614040 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b39096-27b6-4d65-84a7-b7b4bbe79dd0 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:40.614312 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b39096-27b6-4d65-84a7-b7b4bbe79dd0 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:40.614587 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b39096-27b6-4d65-84a7-b7b4bbe79dd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:40.646370 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-e5b39096-27b6-4d65-84a7-b7b4bbe79dd0 None None] MFA Rules not processed for user `12f4cebe3aa74c718651a34613244dac`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:40:40.647656 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-e5b39096-27b6-4d65-84a7-b7b4bbe79dd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:40.728685 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1180/2369] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:40:40 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:40:40.734723 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e649973d-2143-4f37-912e-ff6ebdfb871e admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:40.783924 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:40:40.809849 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e649973d-2143-4f37-912e-ff6ebdfb871e admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:40.815548 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3019ca-8553-4f6c-90cb-56fc4b2dccda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:40.816154 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3019ca-8553-4f6c-90cb-56fc4b2dccda admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:40.816426 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3019ca-8553-4f6c-90cb-56fc4b2dccda admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:40.816692 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3019ca-8553-4f6c-90cb-56fc4b2dccda admin admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:40.817313 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3019ca-8553-4f6c-90cb-56fc4b2dccda admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:40.819216 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3019ca-8553-4f6c-90cb-56fc4b2dccda admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:40.846797 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1190/2370] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:40:40 2022] POST /identity/v3/projects => generated 387 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:40:40.850929 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b39096-27b6-4d65-84a7-b7b4bbe79dd0 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:40.861988 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b39096-27b6-4d65-84a7-b7b4bbe79dd0 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:40.925850 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb29be8-0f94-4ce2-8a45-564855bdf8c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:40.926470 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb29be8-0f94-4ce2-8a45-564855bdf8c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:40.926749 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb29be8-0f94-4ce2-8a45-564855bdf8c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:40.927017 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb29be8-0f94-4ce2-8a45-564855bdf8c0 admin admin] PATH_INFO: `/v3/users` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:40.927815 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb29be8-0f94-4ce2-8a45-564855bdf8c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:40.929922 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb29be8-0f94-4ce2-8a45-564855bdf8c0 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:40.953478 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1181/2371] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:40:40 2022] POST /identity/v3/users => generated 300 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:40:40.956968 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3019ca-8553-4f6c-90cb-56fc4b2dccda admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:40.967564 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3019ca-8553-4f6c-90cb-56fc4b2dccda admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:40.972540 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee88d2fe-53b9-47a9-b500-09f46bb6721d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:40.973134 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee88d2fe-53b9-47a9-b500-09f46bb6721d admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:40.973407 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee88d2fe-53b9-47a9-b500-09f46bb6721d admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:40.973705 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee88d2fe-53b9-47a9-b500-09f46bb6721d admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:40.974329 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee88d2fe-53b9-47a9-b500-09f46bb6721d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:40.975959 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee88d2fe-53b9-47a9-b500-09f46bb6721d admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:40.991295 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1191/2372] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:40:40 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:40:40.996019 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb29be8-0f94-4ce2-8a45-564855bdf8c0 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:41.005738 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb29be8-0f94-4ce2-8a45-564855bdf8c0 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:41.010971 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd0d5c9-bf38-4c05-961d-65f189d3d4b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:41.011794 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd0d5c9-bf38-4c05-961d-65f189d3d4b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:41.011907 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd0d5c9-bf38-4c05-961d-65f189d3d4b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:41.012165 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd0d5c9-bf38-4c05-961d-65f189d3d4b0 admin admin] PATH_INFO: `/v3/projects/05e7172561c34bbaa21055a0eccc469c/users/43406e7f329a43a8945f4f16d828f7f1/roles/176c07b4fe3b4537bdfeb4be21b379c5` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:41.032071 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd0d5c9-bf38-4c05-961d-65f189d3d4b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05e7172561c34bbaa21055a0eccc469c, user_id=43406e7f329a43a8945f4f16d828f7f1, role_id=176c07b4fe3b4537bdfeb4be21b379c5)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:41.034553 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd0d5c9-bf38-4c05-961d-65f189d3d4b0 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:41.048279 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1182/2373] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:40:40 2022] PUT /identity/v3/projects/05e7172561c34bbaa21055a0eccc469c/users/43406e7f329a43a8945f4f16d828f7f1/roles/176c07b4fe3b4537bdfeb4be21b379c5 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:40:41.052268 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee88d2fe-53b9-47a9-b500-09f46bb6721d admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:41.062678 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee88d2fe-53b9-47a9-b500-09f46bb6721d admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:41.125402 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60bf20b-7c0b-4203-a56a-7bbf75dd918e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:41.126102 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60bf20b-7c0b-4203-a56a-7bbf75dd918e admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:41.126386 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60bf20b-7c0b-4203-a56a-7bbf75dd918e admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:41.126661 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60bf20b-7c0b-4203-a56a-7bbf75dd918e admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:41.127306 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60bf20b-7c0b-4203-a56a-7bbf75dd918e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:41.128924 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60bf20b-7c0b-4203-a56a-7bbf75dd918e admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:41.140145 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1192/2374] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:40:41 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:40:41.144354 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd0d5c9-bf38-4c05-961d-65f189d3d4b0 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:41.156121 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd0d5c9-bf38-4c05-961d-65f189d3d4b0 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:41.161156 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dca6019-00ac-4e35-b26e-bfd92ffce8d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:41.161968 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca6019-00ac-4e35-b26e-bfd92ffce8d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:41.162346 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca6019-00ac-4e35-b26e-bfd92ffce8d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:41.162616 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca6019-00ac-4e35-b26e-bfd92ffce8d8 admin admin] PATH_INFO: `/v3/projects/05e7172561c34bbaa21055a0eccc469c/users/43406e7f329a43a8945f4f16d828f7f1/roles/176c07b4fe3b4537bdfeb4be21b379c5` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:41.164855 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dca6019-00ac-4e35-b26e-bfd92ffce8d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05e7172561c34bbaa21055a0eccc469c, user_id=43406e7f329a43a8945f4f16d828f7f1, role_id=176c07b4fe3b4537bdfeb4be21b379c5)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:41.168136 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dca6019-00ac-4e35-b26e-bfd92ffce8d8 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:41.179217 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1183/2375] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:40:41 2022] PUT /identity/v3/projects/05e7172561c34bbaa21055a0eccc469c/users/43406e7f329a43a8945f4f16d828f7f1/roles/176c07b4fe3b4537bdfeb4be21b379c5 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:40:41.190661 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038f49ac-f193-41a4-b14a-5b64756e5068 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:41.190961 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038f49ac-f193-41a4-b14a-5b64756e5068 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:41.191262 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038f49ac-f193-41a4-b14a-5b64756e5068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:41.220440 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-038f49ac-f193-41a4-b14a-5b64756e5068 None None] MFA Rules not processed for user `43406e7f329a43a8945f4f16d828f7f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:40:41.221622 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-038f49ac-f193-41a4-b14a-5b64756e5068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:41.287707 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1193/2376] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:40:41 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 31 12:40:41.298469 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dca6019-00ac-4e35-b26e-bfd92ffce8d8 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:41.300183 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-9dca6019-00ac-4e35-b26e-bfd92ffce8d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:41.311958 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:40:41.362381 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dca6019-00ac-4e35-b26e-bfd92ffce8d8 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:41.366646 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2831dede-36ef-4fea-9d0e-f292c58b1d9d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d6d7ea077d49e8a66c6c6762499702', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:41.367402 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2831dede-36ef-4fea-9d0e-f292c58b1d9d service cinder] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:41.367709 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2831dede-36ef-4fea-9d0e-f292c58b1d9d service cinder] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:41.368117 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2831dede-36ef-4fea-9d0e-f292c58b1d9d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:41.378359 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2831dede-36ef-4fea-9d0e-f292c58b1d9d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:41.379883 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2831dede-36ef-4fea-9d0e-f292c58b1d9d service cinder] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:41.390539 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1184/2377] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:40:41 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:40:41.528163 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038f49ac-f193-41a4-b14a-5b64756e5068 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:41.561873 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:40:41.591478 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038f49ac-f193-41a4-b14a-5b64756e5068 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:41.595958 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526adf9b-69e8-42fd-810f-ad210999c7b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:41.596585 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526adf9b-69e8-42fd-810f-ad210999c7b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:41.596863 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526adf9b-69e8-42fd-810f-ad210999c7b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:41.597135 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526adf9b-69e8-42fd-810f-ad210999c7b7 admin admin] PATH_INFO: `/v3/users/12f4cebe3aa74c718651a34613244dac` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:41.598240 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526adf9b-69e8-42fd-810f-ad210999c7b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12f4cebe3aa74c718651a34613244dac)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:41.600178 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526adf9b-69e8-42fd-810f-ad210999c7b7 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:41.645972 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-526adf9b-69e8-42fd-810f-ad210999c7b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12f4cebe3aa74c718651a34613244dac'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:41.653882 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-526adf9b-69e8-42fd-810f-ad210999c7b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12f4cebe3aa74c718651a34613244dac'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:41.654512 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-526adf9b-69e8-42fd-810f-ad210999c7b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12f4cebe3aa74c718651a34613244dac'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:41.655135 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-526adf9b-69e8-42fd-810f-ad210999c7b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12f4cebe3aa74c718651a34613244dac'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:41.666844 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-526adf9b-69e8-42fd-810f-ad210999c7b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12f4cebe3aa74c718651a34613244dac'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:41.678634 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1194/2378] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:40:41 2022] DELETE /identity/v3/users/12f4cebe3aa74c718651a34613244dac => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:40:41.683914 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824aed4d-7b0e-4b80-8cf9-d028d545a419 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:41.684194 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824aed4d-7b0e-4b80-8cf9-d028d545a419 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:41.684464 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824aed4d-7b0e-4b80-8cf9-d028d545a419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:41.704408 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-824aed4d-7b0e-4b80-8cf9-d028d545a419 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:40:41.705592 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-824aed4d-7b0e-4b80-8cf9-d028d545a419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:41.763308 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1185/2379] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:40:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 31 12:40:41.773360 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526adf9b-69e8-42fd-810f-ad210999c7b7 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:41.774811 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-526adf9b-69e8-42fd-810f-ad210999c7b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:41.842817 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526adf9b-69e8-42fd-810f-ad210999c7b7 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:41.846537 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3f0efd-ea5f-46a2-a7ae-801bdfa015b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:41.847145 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3f0efd-ea5f-46a2-a7ae-801bdfa015b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:41.847455 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3f0efd-ea5f-46a2-a7ae-801bdfa015b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:41.847846 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3f0efd-ea5f-46a2-a7ae-801bdfa015b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:41.858081 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3f0efd-ea5f-46a2-a7ae-801bdfa015b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:41.859584 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3f0efd-ea5f-46a2-a7ae-801bdfa015b6 service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:41.869126 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1195/2380] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:40:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:40:42.149550 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824aed4d-7b0e-4b80-8cf9-d028d545a419 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:42.151002 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-824aed4d-7b0e-4b80-8cf9-d028d545a419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:42.165822 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824aed4d-7b0e-4b80-8cf9-d028d545a419 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:42.170962 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b08cb1-2492-4470-a913-dafc2ccbb750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:42.171619 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b08cb1-2492-4470-a913-dafc2ccbb750 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:42.171893 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b08cb1-2492-4470-a913-dafc2ccbb750 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:42.172159 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b08cb1-2492-4470-a913-dafc2ccbb750 admin admin] PATH_INFO: `/v3/projects/e5495f09e3804dcda8b5a2cdb2c056aa` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:42.173289 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b08cb1-2492-4470-a913-dafc2ccbb750 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5495f09e3804dcda8b5a2cdb2c056aa)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:42.175244 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b08cb1-2492-4470-a913-dafc2ccbb750 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:42.218033 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3f0efd-ea5f-46a2-a7ae-801bdfa015b6 service neutron] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:42.219950 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-dc3f0efd-ea5f-46a2-a7ae-801bdfa015b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:42.226836 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-69b08cb1-2492-4470-a913-dafc2ccbb750 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5495f09e3804dcda8b5a2cdb2c056aa', 'request_id': 'req-69b08cb1-2492-4470-a913-dafc2ccbb750'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:42.234406 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3f0efd-ea5f-46a2-a7ae-801bdfa015b6 service neutron] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:42.236223 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-69b08cb1-2492-4470-a913-dafc2ccbb750 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5495f09e3804dcda8b5a2cdb2c056aa', 'request_id': 'req-69b08cb1-2492-4470-a913-dafc2ccbb750'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:42.245404 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-69b08cb1-2492-4470-a913-dafc2ccbb750 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5495f09e3804dcda8b5a2cdb2c056aa', 'request_id': 'req-69b08cb1-2492-4470-a913-dafc2ccbb750'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:42.256756 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-69b08cb1-2492-4470-a913-dafc2ccbb750 admin admin] The token cache is being invalidate because project e5495f09e3804dcda8b5a2cdb2c056aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:40:42.257047 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-69b08cb1-2492-4470-a913-dafc2ccbb750 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:42.257718 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-69b08cb1-2492-4470-a913-dafc2ccbb750 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:42.259232 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1186/2381] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:40:42 2022] DELETE /identity/v3/projects/e5495f09e3804dcda8b5a2cdb2c056aa => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:40:42.263137 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b08cb1-2492-4470-a913-dafc2ccbb750 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:42.264552 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-69b08cb1-2492-4470-a913-dafc2ccbb750 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:42.287021 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:40:42.296001 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d54ed28-b42d-4671-b607-4c3e7969271f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:42.296719 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d54ed28-b42d-4671-b607-4c3e7969271f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:42.297047 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d54ed28-b42d-4671-b607-4c3e7969271f admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:42.297659 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d54ed28-b42d-4671-b607-4c3e7969271f admin admin] PATH_INFO: `/v3/users/a70671eaf2e44a9d8f685191948da65f` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:42.299154 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d54ed28-b42d-4671-b607-4c3e7969271f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a70671eaf2e44a9d8f685191948da65f)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:42.301201 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d54ed28-b42d-4671-b607-4c3e7969271f admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:42.309598 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:40:42.337469 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b08cb1-2492-4470-a913-dafc2ccbb750 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:42.343150 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d356a8b7-0e6f-47b9-9f6f-697a71eb513c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:42.343799 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d356a8b7-0e6f-47b9-9f6f-697a71eb513c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:42.344073 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d356a8b7-0e6f-47b9-9f6f-697a71eb513c admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:42.344340 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d356a8b7-0e6f-47b9-9f6f-697a71eb513c admin admin] PATH_INFO: `/v3/users/43406e7f329a43a8945f4f16d828f7f1` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:42.345575 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d356a8b7-0e6f-47b9-9f6f-697a71eb513c admin admin] RBAC: Authorizing `identity:delete_user(user_id=43406e7f329a43a8945f4f16d828f7f1)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:42.347486 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d356a8b7-0e6f-47b9-9f6f-697a71eb513c admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:42.351869 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-2d54ed28-b42d-4671-b607-4c3e7969271f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a70671eaf2e44a9d8f685191948da65f'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:42.359834 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-2d54ed28-b42d-4671-b607-4c3e7969271f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a70671eaf2e44a9d8f685191948da65f'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:42.360575 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-2d54ed28-b42d-4671-b607-4c3e7969271f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a70671eaf2e44a9d8f685191948da65f'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:42.361345 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-2d54ed28-b42d-4671-b607-4c3e7969271f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a70671eaf2e44a9d8f685191948da65f'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:42.374436 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-2d54ed28-b42d-4671-b607-4c3e7969271f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a70671eaf2e44a9d8f685191948da65f'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:42.391478 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1196/2382] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:40:42 2022] DELETE /identity/v3/users/a70671eaf2e44a9d8f685191948da65f => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 31 12:40:42.393214 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7b7832-8c6d-423d-87e4-47ba37ac731e None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:42.393503 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7b7832-8c6d-423d-87e4-47ba37ac731e None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:42.393778 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7b7832-8c6d-423d-87e4-47ba37ac731e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:42.395768 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-d356a8b7-0e6f-47b9-9f6f-697a71eb513c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43406e7f329a43a8945f4f16d828f7f1'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:42.405635 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-d356a8b7-0e6f-47b9-9f6f-697a71eb513c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43406e7f329a43a8945f4f16d828f7f1'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:42.406239 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-d356a8b7-0e6f-47b9-9f6f-697a71eb513c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43406e7f329a43a8945f4f16d828f7f1'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:42.415392 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-3f7b7832-8c6d-423d-87e4-47ba37ac731e None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:40:42.417017 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-3f7b7832-8c6d-423d-87e4-47ba37ac731e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:42.421184 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-d356a8b7-0e6f-47b9-9f6f-697a71eb513c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43406e7f329a43a8945f4f16d828f7f1'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:42.428494 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-d356a8b7-0e6f-47b9-9f6f-697a71eb513c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43406e7f329a43a8945f4f16d828f7f1'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:42.435270 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1187/2383] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:40:42 2022] DELETE /identity/v3/users/43406e7f329a43a8945f4f16d828f7f1 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 31 12:40:42.483147 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1197/2384] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:40:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 31 12:40:42.493091 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d356a8b7-0e6f-47b9-9f6f-697a71eb513c admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:42.494598 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-d356a8b7-0e6f-47b9-9f6f-697a71eb513c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:42.556220 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d356a8b7-0e6f-47b9-9f6f-697a71eb513c admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:42.560503 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f22067-7e6d-4354-b632-43ff37b2db95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:42.561107 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f22067-7e6d-4354-b632-43ff37b2db95 service neutron] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:42.561383 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f22067-7e6d-4354-b632-43ff37b2db95 service neutron] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:42.561647 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f22067-7e6d-4354-b632-43ff37b2db95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:42.627106 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f22067-7e6d-4354-b632-43ff37b2db95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:42.628567 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f22067-7e6d-4354-b632-43ff37b2db95 service neutron] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:42.639701 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1188/2385] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:40:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:40:42.788358 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7b7832-8c6d-423d-87e4-47ba37ac731e None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:42.789943 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-3f7b7832-8c6d-423d-87e4-47ba37ac731e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:42.803032 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7b7832-8c6d-423d-87e4-47ba37ac731e None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:42.807677 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e4dcbd-493d-467f-aeae-6ee4b0577b00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:42.808318 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e4dcbd-493d-467f-aeae-6ee4b0577b00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:42.808593 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e4dcbd-493d-467f-aeae-6ee4b0577b00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:42.808860 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e4dcbd-493d-467f-aeae-6ee4b0577b00 admin admin] PATH_INFO: `/v3/projects/05e7172561c34bbaa21055a0eccc469c` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:42.810059 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e4dcbd-493d-467f-aeae-6ee4b0577b00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05e7172561c34bbaa21055a0eccc469c)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:42.812025 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e4dcbd-493d-467f-aeae-6ee4b0577b00 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:42.863046 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-73e4dcbd-493d-467f-aeae-6ee4b0577b00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05e7172561c34bbaa21055a0eccc469c', 'request_id': 'req-73e4dcbd-493d-467f-aeae-6ee4b0577b00'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:42.869689 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-73e4dcbd-493d-467f-aeae-6ee4b0577b00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05e7172561c34bbaa21055a0eccc469c', 'request_id': 'req-73e4dcbd-493d-467f-aeae-6ee4b0577b00'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:42.877935 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-73e4dcbd-493d-467f-aeae-6ee4b0577b00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05e7172561c34bbaa21055a0eccc469c', 'request_id': 'req-73e4dcbd-493d-467f-aeae-6ee4b0577b00'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:42.885120 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:40:42.892350 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-73e4dcbd-493d-467f-aeae-6ee4b0577b00 admin admin] The token cache is being invalidate because project 05e7172561c34bbaa21055a0eccc469c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:40:42.893133 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-73e4dcbd-493d-467f-aeae-6ee4b0577b00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:42.894193 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-73e4dcbd-493d-467f-aeae-6ee4b0577b00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:42.895783 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1198/2386] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:40:42 2022] DELETE /identity/v3/projects/05e7172561c34bbaa21055a0eccc469c => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 31 12:40:42.900658 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be671986-0cb1-48a8-91f5-43f2135133b5 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:42.901046 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be671986-0cb1-48a8-91f5-43f2135133b5 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:42.901490 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be671986-0cb1-48a8-91f5-43f2135133b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:42.910960 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:40:42.926004 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-be671986-0cb1-48a8-91f5-43f2135133b5 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:40:42.928256 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-be671986-0cb1-48a8-91f5-43f2135133b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:42.995966 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e4dcbd-493d-467f-aeae-6ee4b0577b00 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:42.998000 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-73e4dcbd-493d-467f-aeae-6ee4b0577b00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:43.007769 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1189/2387] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:40:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 31 12:40:43.016377 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7b21c6-e9bb-491f-868f-bee5d5d95725 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:43.016799 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7b21c6-e9bb-491f-868f-bee5d5d95725 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:43.016926 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7b21c6-e9bb-491f-868f-bee5d5d95725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:43.021939 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e4dcbd-493d-467f-aeae-6ee4b0577b00 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:43.036990 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ac974d-b399-42db-964f-e100aa41da8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:43.037581 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ac974d-b399-42db-964f-e100aa41da8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:43.037890 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ac974d-b399-42db-964f-e100aa41da8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:43.038159 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ac974d-b399-42db-964f-e100aa41da8f admin admin] PATH_INFO: `/v3/projects/2a757c55f81c4d14a9e17a67a2f17d56` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:43.039783 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ac974d-b399-42db-964f-e100aa41da8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a757c55f81c4d14a9e17a67a2f17d56)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:43.043208 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ac974d-b399-42db-964f-e100aa41da8f admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:43.046730 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-1d7b21c6-e9bb-491f-868f-bee5d5d95725 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:40:43.047971 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-1d7b21c6-e9bb-491f-868f-bee5d5d95725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:43.061778 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1190/2388] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:40:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 31 12:40:43.063413 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7b21c6-e9bb-491f-868f-bee5d5d95725 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:43.073202 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7b21c6-e9bb-491f-868f-bee5d5d95725 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:43.082582 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc1f075-a721-4f41-ab52-4170446de532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:43.083445 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc1f075-a721-4f41-ab52-4170446de532 admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:43.083722 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc1f075-a721-4f41-ab52-4170446de532 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:43.084145 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc1f075-a721-4f41-ab52-4170446de532 admin admin] PATH_INFO: `/v3/domains` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:43.084804 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc1f075-a721-4f41-ab52-4170446de532 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:40:43.085110 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc1f075-a721-4f41-ab52-4170446de532 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:43.086775 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc1f075-a721-4f41-ab52-4170446de532 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:43.097702 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-a7ac974d-b399-42db-964f-e100aa41da8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a757c55f81c4d14a9e17a67a2f17d56', 'request_id': 'req-a7ac974d-b399-42db-964f-e100aa41da8f'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:43.100443 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1191/2389] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Oct 31 12:40:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 31 12:40:43.105357 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-a7ac974d-b399-42db-964f-e100aa41da8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a757c55f81c4d14a9e17a67a2f17d56', 'request_id': 'req-a7ac974d-b399-42db-964f-e100aa41da8f'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:43.105821 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc1f075-a721-4f41-ab52-4170446de532 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:43.113789 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-a7ac974d-b399-42db-964f-e100aa41da8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a757c55f81c4d14a9e17a67a2f17d56', 'request_id': 'req-a7ac974d-b399-42db-964f-e100aa41da8f'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:43.117007 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc1f075-a721-4f41-ab52-4170446de532 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:43.126422 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-a7ac974d-b399-42db-964f-e100aa41da8f admin admin] The token cache is being invalidate because project 2a757c55f81c4d14a9e17a67a2f17d56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:40:43.126713 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-a7ac974d-b399-42db-964f-e100aa41da8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:43.127555 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-a7ac974d-b399-42db-964f-e100aa41da8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:43.129199 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1199/2390] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:40:42 2022] DELETE /identity/v3/projects/2a757c55f81c4d14a9e17a67a2f17d56 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 31 12:40:43.134171 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24032dc4-0d7d-425c-b890-fbf06f0581e0 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:43.134691 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24032dc4-0d7d-425c-b890-fbf06f0581e0 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:43.135034 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24032dc4-0d7d-425c-b890-fbf06f0581e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:43.157629 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-24032dc4-0d7d-425c-b890-fbf06f0581e0 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:40:43.158860 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-24032dc4-0d7d-425c-b890-fbf06f0581e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:43.186966 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975b1ec3-3989-44de-a13f-734fa0aa24f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:43.187635 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975b1ec3-3989-44de-a13f-734fa0aa24f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:43.187890 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975b1ec3-3989-44de-a13f-734fa0aa24f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:43.188159 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975b1ec3-3989-44de-a13f-734fa0aa24f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:43.188784 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975b1ec3-3989-44de-a13f-734fa0aa24f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:43.190670 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975b1ec3-3989-44de-a13f-734fa0aa24f9 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:43.222614 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1192/2391] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:40:43 2022] POST /identity/v3/projects => generated 375 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 31 12:40:43.227523 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975b1ec3-3989-44de-a13f-734fa0aa24f9 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:43.229811 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-975b1ec3-3989-44de-a13f-734fa0aa24f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:43.242096 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1200/2392] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:40:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 31 12:40:43.315147 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975b1ec3-3989-44de-a13f-734fa0aa24f9 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:43.317792 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d130097-c8c9-4c1b-b6da-1b70b3491253 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:43.318278 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d130097-c8c9-4c1b-b6da-1b70b3491253 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:43.318412 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d130097-c8c9-4c1b-b6da-1b70b3491253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:43.321112 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439fe5e0-51c3-4d20-9f6b-e1af56640f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:43.321878 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439fe5e0-51c3-4d20-9f6b-e1af56640f89 admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:43.322281 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439fe5e0-51c3-4d20-9f6b-e1af56640f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:43.322687 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439fe5e0-51c3-4d20-9f6b-e1af56640f89 admin admin] PATH_INFO: `/v3/users` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:43.323397 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439fe5e0-51c3-4d20-9f6b-e1af56640f89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:43.325718 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439fe5e0-51c3-4d20-9f6b-e1af56640f89 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:43.349780 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:40:43.349958 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-2d130097-c8c9-4c1b-b6da-1b70b3491253 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:40:43.351472 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-2d130097-c8c9-4c1b-b6da-1b70b3491253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:43.361290 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1193/2393] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:40:43 2022] POST /identity/v3/users => generated 288 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 31 12:40:43.363226 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1201/2394] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:40:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:40:43.366438 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439fe5e0-51c3-4d20-9f6b-e1af56640f89 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:43.366582 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d130097-c8c9-4c1b-b6da-1b70b3491253 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:43.377367 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d130097-c8c9-4c1b-b6da-1b70b3491253 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:43.383272 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ab3d3b-4ba1-400a-9554-a2119b046c23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:43.383880 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ab3d3b-4ba1-400a-9554-a2119b046c23 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:43.384156 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ab3d3b-4ba1-400a-9554-a2119b046c23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:43.384422 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ab3d3b-4ba1-400a-9554-a2119b046c23 admin admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:43.385056 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ab3d3b-4ba1-400a-9554-a2119b046c23 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:40:43.385871 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ab3d3b-4ba1-400a-9554-a2119b046c23 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:43.387187 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439fe5e0-51c3-4d20-9f6b-e1af56640f89 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:43.387573 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ab3d3b-4ba1-400a-9554-a2119b046c23 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:43.393552 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc43af16-27b7-4b52-b9e2-bfb8e9ab9b8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:43.394285 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc43af16-27b7-4b52-b9e2-bfb8e9ab9b8d admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:43.394653 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc43af16-27b7-4b52-b9e2-bfb8e9ab9b8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:43.395012 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc43af16-27b7-4b52-b9e2-bfb8e9ab9b8d admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:43.395878 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc43af16-27b7-4b52-b9e2-bfb8e9ab9b8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:43.397584 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc43af16-27b7-4b52-b9e2-bfb8e9ab9b8d admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:43.412640 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1202/2395] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Oct 31 12:40:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:40:43.417920 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1194/2396] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:40:43 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:40:43.419183 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc43af16-27b7-4b52-b9e2-bfb8e9ab9b8d admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:43.419710 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ab3d3b-4ba1-400a-9554-a2119b046c23 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:43.429931 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ab3d3b-4ba1-400a-9554-a2119b046c23 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:43.430480 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc43af16-27b7-4b52-b9e2-bfb8e9ab9b8d admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:43.434911 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4755386-07c1-48ce-806a-5ca84ca0949c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:43.435150 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ff7933-a371-47ef-88aa-bd1d139585ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:43.435543 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4755386-07c1-48ce-806a-5ca84ca0949c admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:43.435762 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ff7933-a371-47ef-88aa-bd1d139585ec admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:43.435864 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4755386-07c1-48ce-806a-5ca84ca0949c admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:43.436039 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ff7933-a371-47ef-88aa-bd1d139585ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:43.436156 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4755386-07c1-48ce-806a-5ca84ca0949c admin admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:43.436306 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ff7933-a371-47ef-88aa-bd1d139585ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:43.436674 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4755386-07c1-48ce-806a-5ca84ca0949c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:43.436937 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ff7933-a371-47ef-88aa-bd1d139585ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:43.438546 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ff7933-a371-47ef-88aa-bd1d139585ec admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:43.438673 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4755386-07c1-48ce-806a-5ca84ca0949c admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:43.450850 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1195/2397] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:40:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:40:43.454990 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ff7933-a371-47ef-88aa-bd1d139585ec admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:43.466512 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ff7933-a371-47ef-88aa-bd1d139585ec admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:43.468263 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1203/2398] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:40:43 2022] POST /identity/v3/projects => generated 369 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 31 12:40:43.472443 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4755386-07c1-48ce-806a-5ca84ca0949c admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:43.517529 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:40:43.525662 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03c2283-223d-4b37-b550-b50bcbb3409a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:43.527020 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03c2283-223d-4b37-b550-b50bcbb3409a admin admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:43.527323 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03c2283-223d-4b37-b550-b50bcbb3409a admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:43.527595 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03c2283-223d-4b37-b550-b50bcbb3409a admin admin] PATH_INFO: `/v3/projects/4232b60f28804ed09664b198ab7fa8f8/users/346a4dfe4e384f45b77023863849eef1/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:43.540393 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4755386-07c1-48ce-806a-5ca84ca0949c admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:43.543451 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03c2283-223d-4b37-b550-b50bcbb3409a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4232b60f28804ed09664b198ab7fa8f8, user_id=346a4dfe4e384f45b77023863849eef1, role_id=ca1a5695f8bd499aa0b8cedaeecd2cc6)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:43.545245 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909b64d4-e533-497a-87e7-82420360d0f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:43.545887 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909b64d4-e533-497a-87e7-82420360d0f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:43.546103 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03c2283-223d-4b37-b550-b50bcbb3409a admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:43.546223 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909b64d4-e533-497a-87e7-82420360d0f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:43.546423 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909b64d4-e533-497a-87e7-82420360d0f4 admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:43.547045 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909b64d4-e533-497a-87e7-82420360d0f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:43.548975 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909b64d4-e533-497a-87e7-82420360d0f4 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:43.559445 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1196/2399] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:40:43 2022] PUT /identity/v3/projects/4232b60f28804ed09664b198ab7fa8f8/users/346a4dfe4e384f45b77023863849eef1/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 31 12:40:43.570125 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92500879-57cd-4b65-8637-34c331a391a0 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:43.570408 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92500879-57cd-4b65-8637-34c331a391a0 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:43.570679 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92500879-57cd-4b65-8637-34c331a391a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:43.574167 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1204/2400] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:40:43 2022] POST /identity/v3/users => generated 285 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 31 12:40:43.578062 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909b64d4-e533-497a-87e7-82420360d0f4 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:43.587507 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909b64d4-e533-497a-87e7-82420360d0f4 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:43.601923 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-92500879-57cd-4b65-8637-34c331a391a0 None None] MFA Rules not processed for user `346a4dfe4e384f45b77023863849eef1`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:40:43.603133 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-92500879-57cd-4b65-8637-34c331a391a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:43.645304 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2437af-1b09-460b-8144-60abe6d3deec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:43.646083 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2437af-1b09-460b-8144-60abe6d3deec admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:43.646407 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2437af-1b09-460b-8144-60abe6d3deec admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:43.646923 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2437af-1b09-460b-8144-60abe6d3deec admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:43.647315 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2437af-1b09-460b-8144-60abe6d3deec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:43.648971 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2437af-1b09-460b-8144-60abe6d3deec admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:43.661832 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1205/2401] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:40:43 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 31 12:40:43.663195 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2437af-1b09-460b-8144-60abe6d3deec admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:43.673602 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2437af-1b09-460b-8144-60abe6d3deec admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:43.675110 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1197/2402] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:40:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 31 12:40:43.680678 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1076f209-a1c8-40ab-9b6b-d644941009d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:43.681286 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1076f209-a1c8-40ab-9b6b-d644941009d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:43.681606 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1076f209-a1c8-40ab-9b6b-d644941009d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:43.681827 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1076f209-a1c8-40ab-9b6b-d644941009d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:43.682538 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1076f209-a1c8-40ab-9b6b-d644941009d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:43.683042 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92500879-57cd-4b65-8637-34c331a391a0 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:43.684301 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1076f209-a1c8-40ab-9b6b-d644941009d9 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:43.694261 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92500879-57cd-4b65-8637-34c331a391a0 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:43.695476 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1206/2403] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:40:43 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 31 12:40:43.699575 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1076f209-a1c8-40ab-9b6b-d644941009d9 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:43.702087 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4965ee4d-0248-49e6-a288-745f9c5448dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:43.702839 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4965ee4d-0248-49e6-a288-745f9c5448dd admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:43.703117 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4965ee4d-0248-49e6-a288-745f9c5448dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:43.703411 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4965ee4d-0248-49e6-a288-745f9c5448dd admin admin] PATH_INFO: `/v3/projects` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:43.704032 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4965ee4d-0248-49e6-a288-745f9c5448dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:43.705922 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4965ee4d-0248-49e6-a288-745f9c5448dd admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:43.709101 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1076f209-a1c8-40ab-9b6b-d644941009d9 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:43.714257 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e39b92-909b-4266-ad32-00114015973e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:43.714900 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e39b92-909b-4266-ad32-00114015973e admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:43.715200 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e39b92-909b-4266-ad32-00114015973e admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:43.715472 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e39b92-909b-4266-ad32-00114015973e admin admin] PATH_INFO: `/v3/projects/7947bfb1d9264cd9a226695ef8f8223c/users/f0f42e37e20749738f302ff2defad3e8/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:43.731986 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e39b92-909b-4266-ad32-00114015973e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7947bfb1d9264cd9a226695ef8f8223c, user_id=f0f42e37e20749738f302ff2defad3e8, role_id=ca1a5695f8bd499aa0b8cedaeecd2cc6)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:43.737183 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1198/2404] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:40:43 2022] POST /identity/v3/projects => generated 375 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:40:43.737354 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e39b92-909b-4266-ad32-00114015973e admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:43.738100 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4965ee4d-0248-49e6-a288-745f9c5448dd admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:43.748541 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1207/2405] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:40:43 2022] PUT /identity/v3/projects/7947bfb1d9264cd9a226695ef8f8223c/users/f0f42e37e20749738f302ff2defad3e8/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 31 12:40:43.758389 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b64a85-5f70-470c-9209-4182bc5113a7 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:43.758773 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b64a85-5f70-470c-9209-4182bc5113a7 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:43.759486 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b64a85-5f70-470c-9209-4182bc5113a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:43.769456 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:40:43.787935 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-d0b64a85-5f70-470c-9209-4182bc5113a7 None None] MFA Rules not processed for user `f0f42e37e20749738f302ff2defad3e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:40:43.789129 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-d0b64a85-5f70-470c-9209-4182bc5113a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:43.802742 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4965ee4d-0248-49e6-a288-745f9c5448dd admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:43.857592 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1208/2406] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:40:43 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 31 12:40:43.859010 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6163e3-2ef8-4e65-ad05-0eed033f94b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:43.859648 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6163e3-2ef8-4e65-ad05-0eed033f94b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:43.859927 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6163e3-2ef8-4e65-ad05-0eed033f94b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:43.860198 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6163e3-2ef8-4e65-ad05-0eed033f94b4 admin admin] PATH_INFO: `/v3/users` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:43.860815 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6163e3-2ef8-4e65-ad05-0eed033f94b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:43.863293 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6163e3-2ef8-4e65-ad05-0eed033f94b4 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:43.871377 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b64a85-5f70-470c-9209-4182bc5113a7 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:43.873783 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-d0b64a85-5f70-470c-9209-4182bc5113a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:43.887849 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1199/2407] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:40:43 2022] POST /identity/v3/users => generated 294 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 31 12:40:43.888883 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6163e3-2ef8-4e65-ad05-0eed033f94b4 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:43.898635 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6163e3-2ef8-4e65-ad05-0eed033f94b4 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:43.902664 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:40:43.903533 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f6efaf-abdc-466f-97d3-b0416613a72b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:43.904111 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f6efaf-abdc-466f-97d3-b0416613a72b admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:43.904387 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f6efaf-abdc-466f-97d3-b0416613a72b admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:43.904644 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f6efaf-abdc-466f-97d3-b0416613a72b admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:43.905267 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f6efaf-abdc-466f-97d3-b0416613a72b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:43.906833 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f6efaf-abdc-466f-97d3-b0416613a72b admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:43.917456 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1200/2408] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:40:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 31 12:40:43.921746 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f6efaf-abdc-466f-97d3-b0416613a72b admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:43.931308 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f6efaf-abdc-466f-97d3-b0416613a72b admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:43.936605 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fefae4-f5fe-43fd-aa8c-91c08667d7e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:43.937237 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fefae4-f5fe-43fd-aa8c-91c08667d7e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:43.937516 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fefae4-f5fe-43fd-aa8c-91c08667d7e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:43.937791 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fefae4-f5fe-43fd-aa8c-91c08667d7e1 admin admin] PATH_INFO: `/v3/projects/891a9ae5455a4b44beb34f0c9a70730b/users/164996a63340476ba3958343601056d6/roles/176c07b4fe3b4537bdfeb4be21b379c5` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:43.939503 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b64a85-5f70-470c-9209-4182bc5113a7 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:43.943278 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e107704-f43f-4aee-be53-a779efa4fb59 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4feaa01c5ed64b65b10d4dece823702c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:43.943894 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e107704-f43f-4aee-be53-a779efa4fb59 service glance] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:43.944173 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e107704-f43f-4aee-be53-a779efa4fb59 service glance] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:43.944446 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e107704-f43f-4aee-be53-a779efa4fb59 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:43.953291 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fefae4-f5fe-43fd-aa8c-91c08667d7e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=891a9ae5455a4b44beb34f0c9a70730b, user_id=164996a63340476ba3958343601056d6, role_id=176c07b4fe3b4537bdfeb4be21b379c5)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:43.955014 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e107704-f43f-4aee-be53-a779efa4fb59 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:43.956064 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fefae4-f5fe-43fd-aa8c-91c08667d7e1 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:43.956461 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e107704-f43f-4aee-be53-a779efa4fb59 service glance] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:43.968904 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1209/2409] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:40:43 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 31 12:40:43.970845 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1201/2410] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:40:43 2022] PUT /identity/v3/projects/891a9ae5455a4b44beb34f0c9a70730b/users/164996a63340476ba3958343601056d6/roles/176c07b4fe3b4537bdfeb4be21b379c5 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 31 12:40:43.974649 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e107704-f43f-4aee-be53-a779efa4fb59 service glance] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:43.986401 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e107704-f43f-4aee-be53-a779efa4fb59 service glance] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:44.061647 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5c7d62-455b-4bd7-aa44-6f2c4e8ed974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:44.062254 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5c7d62-455b-4bd7-aa44-6f2c4e8ed974 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:44.062664 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5c7d62-455b-4bd7-aa44-6f2c4e8ed974 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:44.062943 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5c7d62-455b-4bd7-aa44-6f2c4e8ed974 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:44.063614 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5c7d62-455b-4bd7-aa44-6f2c4e8ed974 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:44.065193 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5c7d62-455b-4bd7-aa44-6f2c4e8ed974 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:44.078841 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1210/2411] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:40:43 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 31 12:40:44.079961 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fefae4-f5fe-43fd-aa8c-91c08667d7e1 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:44.090633 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fefae4-f5fe-43fd-aa8c-91c08667d7e1 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:44.095367 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e70077-930d-4c39-860c-e6775819a17d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:44.096016 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e70077-930d-4c39-860c-e6775819a17d admin admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:44.096294 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e70077-930d-4c39-860c-e6775819a17d admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:44.096567 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e70077-930d-4c39-860c-e6775819a17d admin admin] PATH_INFO: `/v3/projects/891a9ae5455a4b44beb34f0c9a70730b/users/164996a63340476ba3958343601056d6/roles/176c07b4fe3b4537bdfeb4be21b379c5` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:44.098744 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e70077-930d-4c39-860c-e6775819a17d admin admin] RBAC: Authorizing `identity:create_grant(project_id=891a9ae5455a4b44beb34f0c9a70730b, user_id=164996a63340476ba3958343601056d6, role_id=176c07b4fe3b4537bdfeb4be21b379c5)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:44.101376 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e70077-930d-4c39-860c-e6775819a17d admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:44.112457 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1202/2412] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:40:44 2022] PUT /identity/v3/projects/891a9ae5455a4b44beb34f0c9a70730b/users/164996a63340476ba3958343601056d6/roles/176c07b4fe3b4537bdfeb4be21b379c5 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:40:44.124901 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4bb250-f0ce-41e8-b9b2-d8caff320542 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:44.125186 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4bb250-f0ce-41e8-b9b2-d8caff320542 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:44.125457 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4bb250-f0ce-41e8-b9b2-d8caff320542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:44.153844 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-2c4bb250-f0ce-41e8-b9b2-d8caff320542 None None] MFA Rules not processed for user `164996a63340476ba3958343601056d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:40:44.155069 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-2c4bb250-f0ce-41e8-b9b2-d8caff320542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:44.226270 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1211/2413] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:40:44 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:40:44.239113 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e70077-930d-4c39-860c-e6775819a17d admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:44.241047 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-53e70077-930d-4c39-860c-e6775819a17d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:44.306570 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e70077-930d-4c39-860c-e6775819a17d admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:44.310893 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ced6c9-6540-4415-ac11-176ad210eefc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d6d7ea077d49e8a66c6c6762499702', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:44.311762 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ced6c9-6540-4415-ac11-176ad210eefc service cinder] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:44.312184 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ced6c9-6540-4415-ac11-176ad210eefc service cinder] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:44.312562 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ced6c9-6540-4415-ac11-176ad210eefc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:44.352649 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:40:44.388802 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ced6c9-6540-4415-ac11-176ad210eefc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:44.390231 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ced6c9-6540-4415-ac11-176ad210eefc service cinder] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:44.400327 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1203/2414] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:40:44 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:40:46.993795 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4bb250-f0ce-41e8-b9b2-d8caff320542 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:47.044522 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:40:47.099687 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4bb250-f0ce-41e8-b9b2-d8caff320542 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:47.104480 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc7e8c4-b3b6-402f-81f1-1d9b06711e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:47.105245 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc7e8c4-b3b6-402f-81f1-1d9b06711e6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:47.105705 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc7e8c4-b3b6-402f-81f1-1d9b06711e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:47.105969 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc7e8c4-b3b6-402f-81f1-1d9b06711e6f admin admin] PATH_INFO: `/v3/users/346a4dfe4e384f45b77023863849eef1` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:47.107235 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc7e8c4-b3b6-402f-81f1-1d9b06711e6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=346a4dfe4e384f45b77023863849eef1)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:47.109182 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc7e8c4-b3b6-402f-81f1-1d9b06711e6f admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:47.157890 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-0dc7e8c4-b3b6-402f-81f1-1d9b06711e6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '346a4dfe4e384f45b77023863849eef1'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:47.165278 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-0dc7e8c4-b3b6-402f-81f1-1d9b06711e6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '346a4dfe4e384f45b77023863849eef1'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:47.166010 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-0dc7e8c4-b3b6-402f-81f1-1d9b06711e6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '346a4dfe4e384f45b77023863849eef1'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:47.166608 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-0dc7e8c4-b3b6-402f-81f1-1d9b06711e6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '346a4dfe4e384f45b77023863849eef1'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:47.179133 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-0dc7e8c4-b3b6-402f-81f1-1d9b06711e6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '346a4dfe4e384f45b77023863849eef1'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:47.193917 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1212/2415] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:40:46 2022] DELETE /identity/v3/users/346a4dfe4e384f45b77023863849eef1 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:40:47.198326 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9751c167-7ca6-4deb-9f22-33708a11c84c None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:47.199903 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9751c167-7ca6-4deb-9f22-33708a11c84c None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:47.199903 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9751c167-7ca6-4deb-9f22-33708a11c84c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:47.257501 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-9751c167-7ca6-4deb-9f22-33708a11c84c None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:40:47.258902 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-9751c167-7ca6-4deb-9f22-33708a11c84c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:47.334341 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1204/2416] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:40:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:40:47.346738 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc7e8c4-b3b6-402f-81f1-1d9b06711e6f admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:47.348556 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-0dc7e8c4-b3b6-402f-81f1-1d9b06711e6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:47.436165 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc7e8c4-b3b6-402f-81f1-1d9b06711e6f admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:47.440377 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f674aa-3421-46d2-ae51-ea0829bb3885 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:47.441120 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f674aa-3421-46d2-ae51-ea0829bb3885 service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:47.441469 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f674aa-3421-46d2-ae51-ea0829bb3885 service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:47.441806 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f674aa-3421-46d2-ae51-ea0829bb3885 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:47.452292 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f674aa-3421-46d2-ae51-ea0829bb3885 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:47.453699 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f674aa-3421-46d2-ae51-ea0829bb3885 service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:47.469201 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1213/2417] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:40:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 31 12:40:47.756558 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9751c167-7ca6-4deb-9f22-33708a11c84c None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:47.757985 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-9751c167-7ca6-4deb-9f22-33708a11c84c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:47.772555 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9751c167-7ca6-4deb-9f22-33708a11c84c None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:47.777800 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af08de47-be0a-4ad3-a3dd-f2e4a5c02f9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:47.778466 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af08de47-be0a-4ad3-a3dd-f2e4a5c02f9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:47.778743 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af08de47-be0a-4ad3-a3dd-f2e4a5c02f9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:47.779017 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af08de47-be0a-4ad3-a3dd-f2e4a5c02f9c admin admin] PATH_INFO: `/v3/projects/4232b60f28804ed09664b198ab7fa8f8` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:47.780423 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af08de47-be0a-4ad3-a3dd-f2e4a5c02f9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4232b60f28804ed09664b198ab7fa8f8)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:47.782397 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af08de47-be0a-4ad3-a3dd-f2e4a5c02f9c admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:47.831721 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-af08de47-be0a-4ad3-a3dd-f2e4a5c02f9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4232b60f28804ed09664b198ab7fa8f8', 'request_id': 'req-af08de47-be0a-4ad3-a3dd-f2e4a5c02f9c'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:47.837341 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-af08de47-be0a-4ad3-a3dd-f2e4a5c02f9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4232b60f28804ed09664b198ab7fa8f8', 'request_id': 'req-af08de47-be0a-4ad3-a3dd-f2e4a5c02f9c'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:47.844094 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-af08de47-be0a-4ad3-a3dd-f2e4a5c02f9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4232b60f28804ed09664b198ab7fa8f8', 'request_id': 'req-af08de47-be0a-4ad3-a3dd-f2e4a5c02f9c'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:47.855414 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-af08de47-be0a-4ad3-a3dd-f2e4a5c02f9c admin admin] The token cache is being invalidate because project 4232b60f28804ed09664b198ab7fa8f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:40:47.855711 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-af08de47-be0a-4ad3-a3dd-f2e4a5c02f9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:47.856931 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-af08de47-be0a-4ad3-a3dd-f2e4a5c02f9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:47.858397 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1205/2418] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:40:47 2022] DELETE /identity/v3/projects/4232b60f28804ed09664b198ab7fa8f8 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:40:47.862595 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f674aa-3421-46d2-ae51-ea0829bb3885 service neutron] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:47.874228 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f674aa-3421-46d2-ae51-ea0829bb3885 service neutron] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:47.878920 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:40:47.932294 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92428b5-d818-4de8-8c04-1d9ba6e9793d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:47.933500 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92428b5-d818-4de8-8c04-1d9ba6e9793d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:47.933776 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92428b5-d818-4de8-8c04-1d9ba6e9793d admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:47.934044 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92428b5-d818-4de8-8c04-1d9ba6e9793d admin admin] PATH_INFO: `/v3/users/164996a63340476ba3958343601056d6` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:47.935154 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92428b5-d818-4de8-8c04-1d9ba6e9793d admin admin] RBAC: Authorizing `identity:delete_user(user_id=164996a63340476ba3958343601056d6)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:47.937133 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92428b5-d818-4de8-8c04-1d9ba6e9793d admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:47.983956 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-d92428b5-d818-4de8-8c04-1d9ba6e9793d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '164996a63340476ba3958343601056d6'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:47.990739 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-d92428b5-d818-4de8-8c04-1d9ba6e9793d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '164996a63340476ba3958343601056d6'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:47.991363 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-d92428b5-d818-4de8-8c04-1d9ba6e9793d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '164996a63340476ba3958343601056d6'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:47.991962 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-d92428b5-d818-4de8-8c04-1d9ba6e9793d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '164996a63340476ba3958343601056d6'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:48.003738 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-d92428b5-d818-4de8-8c04-1d9ba6e9793d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '164996a63340476ba3958343601056d6'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:48.024374 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1214/2419] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:40:47 2022] DELETE /identity/v3/users/164996a63340476ba3958343601056d6 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:40:48.299944 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af08de47-be0a-4ad3-a3dd-f2e4a5c02f9c admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:48.301573 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-af08de47-be0a-4ad3-a3dd-f2e4a5c02f9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:48.317232 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af08de47-be0a-4ad3-a3dd-f2e4a5c02f9c admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:48.345986 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:40:48.379097 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720a1bf0-a087-4e3c-a655-630a49e6d3f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:48.379755 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720a1bf0-a087-4e3c-a655-630a49e6d3f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:48.380035 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720a1bf0-a087-4e3c-a655-630a49e6d3f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:48.380309 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720a1bf0-a087-4e3c-a655-630a49e6d3f5 admin admin] PATH_INFO: `/v3/projects/891a9ae5455a4b44beb34f0c9a70730b` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:48.381416 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720a1bf0-a087-4e3c-a655-630a49e6d3f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=891a9ae5455a4b44beb34f0c9a70730b)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:48.383343 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720a1bf0-a087-4e3c-a655-630a49e6d3f5 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:48.434755 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-720a1bf0-a087-4e3c-a655-630a49e6d3f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '891a9ae5455a4b44beb34f0c9a70730b', 'request_id': 'req-720a1bf0-a087-4e3c-a655-630a49e6d3f5'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:48.440324 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-720a1bf0-a087-4e3c-a655-630a49e6d3f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '891a9ae5455a4b44beb34f0c9a70730b', 'request_id': 'req-720a1bf0-a087-4e3c-a655-630a49e6d3f5'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:48.447584 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-720a1bf0-a087-4e3c-a655-630a49e6d3f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '891a9ae5455a4b44beb34f0c9a70730b', 'request_id': 'req-720a1bf0-a087-4e3c-a655-630a49e6d3f5'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:48.459643 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-720a1bf0-a087-4e3c-a655-630a49e6d3f5 admin admin] The token cache is being invalidate because project 891a9ae5455a4b44beb34f0c9a70730b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:40:48.459939 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-720a1bf0-a087-4e3c-a655-630a49e6d3f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:48.460577 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-720a1bf0-a087-4e3c-a655-630a49e6d3f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:48.462252 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1206/2420] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:40:48 2022] DELETE /identity/v3/projects/891a9ae5455a4b44beb34f0c9a70730b => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:40:48.468477 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1385f93-70d9-46e2-b5f8-198042152410 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:48.468762 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1385f93-70d9-46e2-b5f8-198042152410 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:48.469342 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1385f93-70d9-46e2-b5f8-198042152410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:48.521070 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-a1385f93-70d9-46e2-b5f8-198042152410 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:40:48.522270 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-a1385f93-70d9-46e2-b5f8-198042152410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:48.582433 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1215/2421] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:40:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:40:48.590890 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05ded65-f5eb-4593-852a-5bcac7d93900 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:48.591192 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05ded65-f5eb-4593-852a-5bcac7d93900 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:48.591464 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05ded65-f5eb-4593-852a-5bcac7d93900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:48.611533 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-c05ded65-f5eb-4593-852a-5bcac7d93900 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:40:48.612853 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-c05ded65-f5eb-4593-852a-5bcac7d93900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:48.622067 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1207/2422] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:40:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:40:48.626004 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1385f93-70d9-46e2-b5f8-198042152410 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:48.632081 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:40:48.637122 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1385f93-70d9-46e2-b5f8-198042152410 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:48.644257 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7a1f82-8567-4356-8db3-9ad0130ddc91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:48.645115 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7a1f82-8567-4356-8db3-9ad0130ddc91 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:48.645793 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7a1f82-8567-4356-8db3-9ad0130ddc91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:48.645932 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7a1f82-8567-4356-8db3-9ad0130ddc91 admin admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:48.646883 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7a1f82-8567-4356-8db3-9ad0130ddc91 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:40:48.647060 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7a1f82-8567-4356-8db3-9ad0130ddc91 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:48.648686 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7a1f82-8567-4356-8db3-9ad0130ddc91 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:48.661138 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1216/2423] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Oct 31 12:40:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:40:48.665624 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05ded65-f5eb-4593-852a-5bcac7d93900 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:48.675827 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05ded65-f5eb-4593-852a-5bcac7d93900 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:48.680845 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ee0608-1562-4c55-9340-3739c5ed1959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:48.681619 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ee0608-1562-4c55-9340-3739c5ed1959 admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:48.681895 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ee0608-1562-4c55-9340-3739c5ed1959 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:48.682180 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ee0608-1562-4c55-9340-3739c5ed1959 admin admin] PATH_INFO: `/v3/projects` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:48.682970 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ee0608-1562-4c55-9340-3739c5ed1959 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:48.685128 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ee0608-1562-4c55-9340-3739c5ed1959 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:48.715711 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1208/2424] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:40:48 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:40:48.720002 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7a1f82-8567-4356-8db3-9ad0130ddc91 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:48.730017 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7a1f82-8567-4356-8db3-9ad0130ddc91 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:48.795647 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bd6413-6b60-4617-9cf6-081aa006670a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:48.796256 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bd6413-6b60-4617-9cf6-081aa006670a admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:48.796534 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bd6413-6b60-4617-9cf6-081aa006670a admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:48.796801 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bd6413-6b60-4617-9cf6-081aa006670a admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:48.797430 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bd6413-6b60-4617-9cf6-081aa006670a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:48.799455 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bd6413-6b60-4617-9cf6-081aa006670a admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:48.822142 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1217/2425] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:40:48 2022] POST /identity/v3/users => generated 293 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:40:48.827003 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ee0608-1562-4c55-9340-3739c5ed1959 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:48.837618 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ee0608-1562-4c55-9340-3739c5ed1959 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:48.844386 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eded4b8-6fc3-4d4b-8d31-ae1b3e2ee29f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:48.845018 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eded4b8-6fc3-4d4b-8d31-ae1b3e2ee29f admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:48.845273 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eded4b8-6fc3-4d4b-8d31-ae1b3e2ee29f admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:48.845541 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eded4b8-6fc3-4d4b-8d31-ae1b3e2ee29f admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:48.846224 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eded4b8-6fc3-4d4b-8d31-ae1b3e2ee29f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:48.847878 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eded4b8-6fc3-4d4b-8d31-ae1b3e2ee29f admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:48.858989 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1209/2426] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:40:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:40:48.863823 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bd6413-6b60-4617-9cf6-081aa006670a admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:48.874081 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bd6413-6b60-4617-9cf6-081aa006670a admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:48.880910 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9fbd62-a1f7-49a9-8060-94eb7b5badbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:48.881709 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9fbd62-a1f7-49a9-8060-94eb7b5badbe admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:48.881992 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9fbd62-a1f7-49a9-8060-94eb7b5badbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:48.882270 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9fbd62-a1f7-49a9-8060-94eb7b5badbe admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:48.882964 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9fbd62-a1f7-49a9-8060-94eb7b5badbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:48.884604 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9fbd62-a1f7-49a9-8060-94eb7b5badbe admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:48.897626 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1218/2427] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:40:48 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:40:48.904523 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eded4b8-6fc3-4d4b-8d31-ae1b3e2ee29f admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:48.914918 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eded4b8-6fc3-4d4b-8d31-ae1b3e2ee29f admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:48.919878 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e31a18d-60f1-4ad0-b0a2-2fd74c4c4140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:48.920523 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e31a18d-60f1-4ad0-b0a2-2fd74c4c4140 admin admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:48.920819 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e31a18d-60f1-4ad0-b0a2-2fd74c4c4140 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:48.921080 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e31a18d-60f1-4ad0-b0a2-2fd74c4c4140 admin admin] PATH_INFO: `/v3/projects/28bdc9dc44b7489faf4021d415e963f6/users/91a570bb4eae439ebb4880aec416e9b6/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:48.937181 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e31a18d-60f1-4ad0-b0a2-2fd74c4c4140 admin admin] RBAC: Authorizing `identity:create_grant(project_id=28bdc9dc44b7489faf4021d415e963f6, user_id=91a570bb4eae439ebb4880aec416e9b6, role_id=ca1a5695f8bd499aa0b8cedaeecd2cc6)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:48.940313 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e31a18d-60f1-4ad0-b0a2-2fd74c4c4140 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:48.954898 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1210/2428] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:40:48 2022] PUT /identity/v3/projects/28bdc9dc44b7489faf4021d415e963f6/users/91a570bb4eae439ebb4880aec416e9b6/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 31 12:40:48.967638 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea48810d-f05f-4ba7-9914-ef6d927dba01 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:48.968295 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea48810d-f05f-4ba7-9914-ef6d927dba01 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:48.968534 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea48810d-f05f-4ba7-9914-ef6d927dba01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:48.999044 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-ea48810d-f05f-4ba7-9914-ef6d927dba01 None None] MFA Rules not processed for user `91a570bb4eae439ebb4880aec416e9b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:40:49.000359 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-ea48810d-f05f-4ba7-9914-ef6d927dba01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:49.083135 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1219/2429] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:40:48 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:40:49.096968 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e31a18d-60f1-4ad0-b0a2-2fd74c4c4140 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:49.099270 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-8e31a18d-60f1-4ad0-b0a2-2fd74c4c4140 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:49.168550 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e31a18d-60f1-4ad0-b0a2-2fd74c4c4140 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:49.172220 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93d03d8-b3f9-4064-b172-e6396c36f9d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d6d7ea077d49e8a66c6c6762499702', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:49.172826 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93d03d8-b3f9-4064-b172-e6396c36f9d5 service cinder] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:49.173105 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93d03d8-b3f9-4064-b172-e6396c36f9d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:49.173378 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93d03d8-b3f9-4064-b172-e6396c36f9d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:49.181545 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93d03d8-b3f9-4064-b172-e6396c36f9d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:49.182996 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93d03d8-b3f9-4064-b172-e6396c36f9d5 service cinder] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:49.186417 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:40:49.193443 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1211/2430] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:40:49 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:40:51.771352 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea48810d-f05f-4ba7-9914-ef6d927dba01 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:51.844036 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea48810d-f05f-4ba7-9914-ef6d927dba01 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:51.844036 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47848854-e90c-448f-9935-794b9d9802af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:51.844332 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47848854-e90c-448f-9935-794b9d9802af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:51.844677 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47848854-e90c-448f-9935-794b9d9802af admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:51.844953 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47848854-e90c-448f-9935-794b9d9802af admin admin] PATH_INFO: `/v3/users/91a570bb4eae439ebb4880aec416e9b6` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:51.846090 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47848854-e90c-448f-9935-794b9d9802af admin admin] RBAC: Authorizing `identity:delete_user(user_id=91a570bb4eae439ebb4880aec416e9b6)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:51.848028 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47848854-e90c-448f-9935-794b9d9802af admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:51.849756 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:40:51.896352 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-47848854-e90c-448f-9935-794b9d9802af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91a570bb4eae439ebb4880aec416e9b6'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:51.903994 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-47848854-e90c-448f-9935-794b9d9802af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91a570bb4eae439ebb4880aec416e9b6'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:51.904536 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-47848854-e90c-448f-9935-794b9d9802af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91a570bb4eae439ebb4880aec416e9b6'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:51.905286 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-47848854-e90c-448f-9935-794b9d9802af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91a570bb4eae439ebb4880aec416e9b6'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:51.918380 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-47848854-e90c-448f-9935-794b9d9802af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91a570bb4eae439ebb4880aec416e9b6'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:51.930536 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1220/2431] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:40:51 2022] DELETE /identity/v3/users/91a570bb4eae439ebb4880aec416e9b6 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:40:51.935115 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c225834-e4da-4908-9059-25f5151fbf76 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:51.935743 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c225834-e4da-4908-9059-25f5151fbf76 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:51.935743 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c225834-e4da-4908-9059-25f5151fbf76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:51.955283 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-2c225834-e4da-4908-9059-25f5151fbf76 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:40:51.956472 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-2c225834-e4da-4908-9059-25f5151fbf76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:52.018797 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1212/2432] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:40:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:40:52.028754 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47848854-e90c-448f-9935-794b9d9802af admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:52.030272 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-47848854-e90c-448f-9935-794b9d9802af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:52.089541 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47848854-e90c-448f-9935-794b9d9802af admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:52.093822 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83daf481-7960-41a1-b1fb-70733485498f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:52.094429 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83daf481-7960-41a1-b1fb-70733485498f service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:52.094709 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83daf481-7960-41a1-b1fb-70733485498f service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:52.094983 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83daf481-7960-41a1-b1fb-70733485498f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:52.104450 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83daf481-7960-41a1-b1fb-70733485498f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:52.105989 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83daf481-7960-41a1-b1fb-70733485498f service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:52.117452 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1221/2433] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:40:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:40:52.393737 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c225834-e4da-4908-9059-25f5151fbf76 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:52.395257 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-2c225834-e4da-4908-9059-25f5151fbf76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:52.407930 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c225834-e4da-4908-9059-25f5151fbf76 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:52.412559 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c26fdc3-f87c-4a4c-9702-d1b2ae858d3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:52.413298 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c26fdc3-f87c-4a4c-9702-d1b2ae858d3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:52.413677 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c26fdc3-f87c-4a4c-9702-d1b2ae858d3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:52.413924 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c26fdc3-f87c-4a4c-9702-d1b2ae858d3a admin admin] PATH_INFO: `/v3/projects/28bdc9dc44b7489faf4021d415e963f6` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:52.414967 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c26fdc3-f87c-4a4c-9702-d1b2ae858d3a admin admin] RBAC: Authorizing `identity:delete_project(project_id=28bdc9dc44b7489faf4021d415e963f6)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:52.416980 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c26fdc3-f87c-4a4c-9702-d1b2ae858d3a admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:52.466123 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-7c26fdc3-f87c-4a4c-9702-d1b2ae858d3a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28bdc9dc44b7489faf4021d415e963f6', 'request_id': 'req-7c26fdc3-f87c-4a4c-9702-d1b2ae858d3a'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:52.472328 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-7c26fdc3-f87c-4a4c-9702-d1b2ae858d3a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28bdc9dc44b7489faf4021d415e963f6', 'request_id': 'req-7c26fdc3-f87c-4a4c-9702-d1b2ae858d3a'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:52.479389 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-7c26fdc3-f87c-4a4c-9702-d1b2ae858d3a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28bdc9dc44b7489faf4021d415e963f6', 'request_id': 'req-7c26fdc3-f87c-4a4c-9702-d1b2ae858d3a'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:52.490315 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-7c26fdc3-f87c-4a4c-9702-d1b2ae858d3a admin admin] The token cache is being invalidate because project 28bdc9dc44b7489faf4021d415e963f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:40:52.490596 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-7c26fdc3-f87c-4a4c-9702-d1b2ae858d3a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:52.491235 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-7c26fdc3-f87c-4a4c-9702-d1b2ae858d3a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:52.492723 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1213/2434] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:40:52 2022] DELETE /identity/v3/projects/28bdc9dc44b7489faf4021d415e963f6 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:40:52.497755 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc60b8f-2c71-4301-9db7-ccffafc71733 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:52.498042 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc60b8f-2c71-4301-9db7-ccffafc71733 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:52.498316 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc60b8f-2c71-4301-9db7-ccffafc71733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:52.521565 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-dcc60b8f-2c71-4301-9db7-ccffafc71733 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:40:52.522848 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-dcc60b8f-2c71-4301-9db7-ccffafc71733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:52.582786 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1222/2435] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:40:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 31 12:40:52.590773 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739c0229-d2f3-4d0c-9e9a-0d5e03a97a1a None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:52.591056 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739c0229-d2f3-4d0c-9e9a-0d5e03a97a1a None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:52.591358 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739c0229-d2f3-4d0c-9e9a-0d5e03a97a1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:52.611902 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-739c0229-d2f3-4d0c-9e9a-0d5e03a97a1a None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:40:52.613081 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-739c0229-d2f3-4d0c-9e9a-0d5e03a97a1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:52.620769 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1214/2436] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:40:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:40:52.625582 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc60b8f-2c71-4301-9db7-ccffafc71733 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:52.635699 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc60b8f-2c71-4301-9db7-ccffafc71733 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:52.640564 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a830b19-89f1-47a2-ad49-a482c7c9c185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:52.641252 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a830b19-89f1-47a2-ad49-a482c7c9c185 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:52.641527 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a830b19-89f1-47a2-ad49-a482c7c9c185 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:52.641837 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a830b19-89f1-47a2-ad49-a482c7c9c185 admin admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:52.642512 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a830b19-89f1-47a2-ad49-a482c7c9c185 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:40:52.643106 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a830b19-89f1-47a2-ad49-a482c7c9c185 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:52.644796 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a830b19-89f1-47a2-ad49-a482c7c9c185 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:52.659418 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1223/2437] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Oct 31 12:40:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:40:52.663908 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739c0229-d2f3-4d0c-9e9a-0d5e03a97a1a None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:52.673886 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739c0229-d2f3-4d0c-9e9a-0d5e03a97a1a None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:52.678485 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0932aa-2a09-4b5d-b505-dc5793459b72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:52.679090 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0932aa-2a09-4b5d-b505-dc5793459b72 admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:52.679396 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0932aa-2a09-4b5d-b505-dc5793459b72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:52.679671 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0932aa-2a09-4b5d-b505-dc5793459b72 admin admin] PATH_INFO: `/v3/projects` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:52.680296 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0932aa-2a09-4b5d-b505-dc5793459b72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:52.682148 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0932aa-2a09-4b5d-b505-dc5793459b72 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:52.713372 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1215/2438] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:40:52 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:40:52.714742 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a830b19-89f1-47a2-ad49-a482c7c9c185 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:52.725625 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a830b19-89f1-47a2-ad49-a482c7c9c185 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:52.731506 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:40:52.799333 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c077e5-ad32-4efe-ac22-df8adce6b39d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:52.799980 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c077e5-ad32-4efe-ac22-df8adce6b39d admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:52.800256 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c077e5-ad32-4efe-ac22-df8adce6b39d admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:52.800527 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c077e5-ad32-4efe-ac22-df8adce6b39d admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:52.801163 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c077e5-ad32-4efe-ac22-df8adce6b39d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:52.803061 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c077e5-ad32-4efe-ac22-df8adce6b39d admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:52.828490 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1224/2439] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:40:52 2022] POST /identity/v3/users => generated 287 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 31 12:40:52.832654 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0932aa-2a09-4b5d-b505-dc5793459b72 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:52.843616 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0932aa-2a09-4b5d-b505-dc5793459b72 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:52.849507 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256af9c2-3882-4e99-aa5a-9625d937b55e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:52.850237 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256af9c2-3882-4e99-aa5a-9625d937b55e admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:52.850596 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256af9c2-3882-4e99-aa5a-9625d937b55e admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:52.853200 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256af9c2-3882-4e99-aa5a-9625d937b55e admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:52.853996 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256af9c2-3882-4e99-aa5a-9625d937b55e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:52.855817 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256af9c2-3882-4e99-aa5a-9625d937b55e admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:52.871148 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1216/2440] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:40:52 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:40:52.872602 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c077e5-ad32-4efe-ac22-df8adce6b39d admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:52.885042 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c077e5-ad32-4efe-ac22-df8adce6b39d admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:52.890603 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc45e62-872d-4d73-ade7-5477d4866ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:52.891456 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc45e62-872d-4d73-ade7-5477d4866ce6 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:52.891734 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc45e62-872d-4d73-ade7-5477d4866ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:52.892001 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc45e62-872d-4d73-ade7-5477d4866ce6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:52.893074 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc45e62-872d-4d73-ade7-5477d4866ce6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:52.894503 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc45e62-872d-4d73-ade7-5477d4866ce6 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:52.905996 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1225/2441] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:40:52 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 31 12:40:52.911201 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256af9c2-3882-4e99-aa5a-9625d937b55e admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:52.922789 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256af9c2-3882-4e99-aa5a-9625d937b55e admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:52.927826 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3386cbb3-a7c2-4aa8-8ccb-0b442cd613d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:52.928595 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3386cbb3-a7c2-4aa8-8ccb-0b442cd613d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:52.928967 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3386cbb3-a7c2-4aa8-8ccb-0b442cd613d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:52.929333 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3386cbb3-a7c2-4aa8-8ccb-0b442cd613d3 admin admin] PATH_INFO: `/v3/projects/2aaba8d0767d4916a7d13de47ddb7f84/users/44b30dbe06204ce5876ad780d4a18305/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:52.945147 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3386cbb3-a7c2-4aa8-8ccb-0b442cd613d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2aaba8d0767d4916a7d13de47ddb7f84, user_id=44b30dbe06204ce5876ad780d4a18305, role_id=ca1a5695f8bd499aa0b8cedaeecd2cc6)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:52.949982 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3386cbb3-a7c2-4aa8-8ccb-0b442cd613d3 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:52.964356 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1217/2442] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:40:52 2022] PUT /identity/v3/projects/2aaba8d0767d4916a7d13de47ddb7f84/users/44b30dbe06204ce5876ad780d4a18305/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 31 12:40:52.975185 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede4fd13-e083-49b0-b521-25c634b37b65 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:52.975474 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede4fd13-e083-49b0-b521-25c634b37b65 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:52.975756 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede4fd13-e083-49b0-b521-25c634b37b65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:53.005609 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-ede4fd13-e083-49b0-b521-25c634b37b65 None None] MFA Rules not processed for user `44b30dbe06204ce5876ad780d4a18305`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:40:53.009225 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-ede4fd13-e083-49b0-b521-25c634b37b65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:53.080338 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1226/2443] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:40:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 31 12:40:53.091437 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3386cbb3-a7c2-4aa8-8ccb-0b442cd613d3 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:53.093111 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-3386cbb3-a7c2-4aa8-8ccb-0b442cd613d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:53.114528 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:40:53.153951 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3386cbb3-a7c2-4aa8-8ccb-0b442cd613d3 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:53.158075 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7076998b-7c62-49be-9a36-6f3faaf37bf2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d6d7ea077d49e8a66c6c6762499702', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:53.158667 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7076998b-7c62-49be-9a36-6f3faaf37bf2 service cinder] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:53.158941 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7076998b-7c62-49be-9a36-6f3faaf37bf2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:53.159239 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7076998b-7c62-49be-9a36-6f3faaf37bf2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:53.167264 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7076998b-7c62-49be-9a36-6f3faaf37bf2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:53.168668 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7076998b-7c62-49be-9a36-6f3faaf37bf2 service cinder] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:53.177345 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1218/2444] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:40:53 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:40:53.202805 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede4fd13-e083-49b0-b521-25c634b37b65 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:53.265232 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede4fd13-e083-49b0-b521-25c634b37b65 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:53.271038 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aee2462-f573-42fc-be07-bb813d800860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:53.271719 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aee2462-f573-42fc-be07-bb813d800860 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:53.271995 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aee2462-f573-42fc-be07-bb813d800860 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:53.272262 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aee2462-f573-42fc-be07-bb813d800860 admin admin] PATH_INFO: `/v3/users/44b30dbe06204ce5876ad780d4a18305` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:53.273459 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aee2462-f573-42fc-be07-bb813d800860 admin admin] RBAC: Authorizing `identity:delete_user(user_id=44b30dbe06204ce5876ad780d4a18305)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:53.275594 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aee2462-f573-42fc-be07-bb813d800860 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:53.297448 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:40:53.325062 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-2aee2462-f573-42fc-be07-bb813d800860 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44b30dbe06204ce5876ad780d4a18305'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:53.332578 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-2aee2462-f573-42fc-be07-bb813d800860 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44b30dbe06204ce5876ad780d4a18305'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:53.333230 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-2aee2462-f573-42fc-be07-bb813d800860 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44b30dbe06204ce5876ad780d4a18305'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:53.333914 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-2aee2462-f573-42fc-be07-bb813d800860 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44b30dbe06204ce5876ad780d4a18305'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:53.348338 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-2aee2462-f573-42fc-be07-bb813d800860 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44b30dbe06204ce5876ad780d4a18305'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:53.361376 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1227/2445] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:40:53 2022] DELETE /identity/v3/users/44b30dbe06204ce5876ad780d4a18305 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 31 12:40:53.365835 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84292146-8b8b-48c3-8e2e-e850b36b2032 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:53.366117 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84292146-8b8b-48c3-8e2e-e850b36b2032 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:53.366384 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84292146-8b8b-48c3-8e2e-e850b36b2032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:53.385530 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-84292146-8b8b-48c3-8e2e-e850b36b2032 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:40:53.386739 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-84292146-8b8b-48c3-8e2e-e850b36b2032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:53.448106 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1219/2446] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:40:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:40:53.458270 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aee2462-f573-42fc-be07-bb813d800860 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:53.459722 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-2aee2462-f573-42fc-be07-bb813d800860 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:53.518765 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aee2462-f573-42fc-be07-bb813d800860 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:53.522367 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15e5ba1-7739-4ba2-89d5-2d9e631bbe62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:53.522971 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15e5ba1-7739-4ba2-89d5-2d9e631bbe62 service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:53.523271 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15e5ba1-7739-4ba2-89d5-2d9e631bbe62 service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:53.523543 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15e5ba1-7739-4ba2-89d5-2d9e631bbe62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:53.532813 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15e5ba1-7739-4ba2-89d5-2d9e631bbe62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:53.534211 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15e5ba1-7739-4ba2-89d5-2d9e631bbe62 service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:53.543807 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1228/2447] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:40:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:40:53.820053 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84292146-8b8b-48c3-8e2e-e850b36b2032 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:53.821466 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-84292146-8b8b-48c3-8e2e-e850b36b2032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:53.834375 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84292146-8b8b-48c3-8e2e-e850b36b2032 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:53.838916 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8856d2-bc35-4d82-aed8-6001c4f01974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:53.839572 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8856d2-bc35-4d82-aed8-6001c4f01974 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:53.839851 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8856d2-bc35-4d82-aed8-6001c4f01974 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:53.840155 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8856d2-bc35-4d82-aed8-6001c4f01974 admin admin] PATH_INFO: `/v3/projects/2aaba8d0767d4916a7d13de47ddb7f84` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:53.841290 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8856d2-bc35-4d82-aed8-6001c4f01974 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2aaba8d0767d4916a7d13de47ddb7f84)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:53.843226 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8856d2-bc35-4d82-aed8-6001c4f01974 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:53.892321 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-cb8856d2-bc35-4d82-aed8-6001c4f01974 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2aaba8d0767d4916a7d13de47ddb7f84', 'request_id': 'req-cb8856d2-bc35-4d82-aed8-6001c4f01974'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:53.898065 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-cb8856d2-bc35-4d82-aed8-6001c4f01974 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2aaba8d0767d4916a7d13de47ddb7f84', 'request_id': 'req-cb8856d2-bc35-4d82-aed8-6001c4f01974'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:53.904858 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-cb8856d2-bc35-4d82-aed8-6001c4f01974 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2aaba8d0767d4916a7d13de47ddb7f84', 'request_id': 'req-cb8856d2-bc35-4d82-aed8-6001c4f01974'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:53.916030 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-cb8856d2-bc35-4d82-aed8-6001c4f01974 admin admin] The token cache is being invalidate because project 2aaba8d0767d4916a7d13de47ddb7f84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:40:53.916316 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-cb8856d2-bc35-4d82-aed8-6001c4f01974 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:53.916962 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-cb8856d2-bc35-4d82-aed8-6001c4f01974 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:53.918557 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1220/2448] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:40:53 2022] DELETE /identity/v3/projects/2aaba8d0767d4916a7d13de47ddb7f84 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 31 12:40:55.801494 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15e5ba1-7739-4ba2-89d5-2d9e631bbe62 service neutron] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:55.802890 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-e15e5ba1-7739-4ba2-89d5-2d9e631bbe62 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:55.816157 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15e5ba1-7739-4ba2-89d5-2d9e631bbe62 service neutron] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:55.873486 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a816248-e73c-4fd6-8402-a57231571fe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:55.874110 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a816248-e73c-4fd6-8402-a57231571fe9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:55.874383 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a816248-e73c-4fd6-8402-a57231571fe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:55.874652 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a816248-e73c-4fd6-8402-a57231571fe9 admin admin] PATH_INFO: `/v3/users/c4b03b84194642e8a0d27ce0edb7a58b` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:55.876277 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a816248-e73c-4fd6-8402-a57231571fe9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4b03b84194642e8a0d27ce0edb7a58b)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:55.878850 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a816248-e73c-4fd6-8402-a57231571fe9 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:55.926144 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:40:55.932363 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-0a816248-e73c-4fd6-8402-a57231571fe9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4b03b84194642e8a0d27ce0edb7a58b'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:55.940419 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-0a816248-e73c-4fd6-8402-a57231571fe9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4b03b84194642e8a0d27ce0edb7a58b'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:55.941288 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-0a816248-e73c-4fd6-8402-a57231571fe9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4b03b84194642e8a0d27ce0edb7a58b'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:55.941942 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-0a816248-e73c-4fd6-8402-a57231571fe9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4b03b84194642e8a0d27ce0edb7a58b'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:55.954129 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-0a816248-e73c-4fd6-8402-a57231571fe9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4b03b84194642e8a0d27ce0edb7a58b'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:55.965987 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1229/2449] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:40:55 2022] DELETE /identity/v3/users/c4b03b84194642e8a0d27ce0edb7a58b => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:40:55.970772 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c61213-39af-4c93-809f-7a8d0d9d3821 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:55.971250 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c61213-39af-4c93-809f-7a8d0d9d3821 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:55.971365 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c61213-39af-4c93-809f-7a8d0d9d3821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:55.991533 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-b3c61213-39af-4c93-809f-7a8d0d9d3821 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:40:55.992793 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-b3c61213-39af-4c93-809f-7a8d0d9d3821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:56.039184 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:40:56.065653 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1221/2450] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:40:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:40:56.075906 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a816248-e73c-4fd6-8402-a57231571fe9 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:56.077310 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-0a816248-e73c-4fd6-8402-a57231571fe9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:56.139450 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a816248-e73c-4fd6-8402-a57231571fe9 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:56.144083 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706336bc-8e5c-4519-a709-2c7870b9673e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:56.144701 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706336bc-8e5c-4519-a709-2c7870b9673e service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:56.144982 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706336bc-8e5c-4519-a709-2c7870b9673e service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:56.145255 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706336bc-8e5c-4519-a709-2c7870b9673e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:56.154488 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706336bc-8e5c-4519-a709-2c7870b9673e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:56.155954 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706336bc-8e5c-4519-a709-2c7870b9673e service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:56.165810 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1230/2451] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:40:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:40:56.441670 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c61213-39af-4c93-809f-7a8d0d9d3821 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:56.443085 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-b3c61213-39af-4c93-809f-7a8d0d9d3821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:56.456519 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c61213-39af-4c93-809f-7a8d0d9d3821 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:56.461144 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6ed103-9e24-4778-8f8e-a3ce83510763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:56.461779 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6ed103-9e24-4778-8f8e-a3ce83510763 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:56.462057 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6ed103-9e24-4778-8f8e-a3ce83510763 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:56.462324 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6ed103-9e24-4778-8f8e-a3ce83510763 admin admin] PATH_INFO: `/v3/projects/ab689a8695914fe3a9004e6603eb3c0f` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:56.463446 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6ed103-9e24-4778-8f8e-a3ce83510763 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab689a8695914fe3a9004e6603eb3c0f)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:56.465317 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6ed103-9e24-4778-8f8e-a3ce83510763 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:56.512481 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-bd6ed103-9e24-4778-8f8e-a3ce83510763 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab689a8695914fe3a9004e6603eb3c0f', 'request_id': 'req-bd6ed103-9e24-4778-8f8e-a3ce83510763'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:56.518723 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-bd6ed103-9e24-4778-8f8e-a3ce83510763 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab689a8695914fe3a9004e6603eb3c0f', 'request_id': 'req-bd6ed103-9e24-4778-8f8e-a3ce83510763'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:56.525378 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-bd6ed103-9e24-4778-8f8e-a3ce83510763 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab689a8695914fe3a9004e6603eb3c0f', 'request_id': 'req-bd6ed103-9e24-4778-8f8e-a3ce83510763'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:56.536819 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-bd6ed103-9e24-4778-8f8e-a3ce83510763 admin admin] The token cache is being invalidate because project ab689a8695914fe3a9004e6603eb3c0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:40:56.537106 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-bd6ed103-9e24-4778-8f8e-a3ce83510763 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:56.537743 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-bd6ed103-9e24-4778-8f8e-a3ce83510763 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:56.539238 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1222/2452] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:40:56 2022] DELETE /identity/v3/projects/ab689a8695914fe3a9004e6603eb3c0f => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:40:56.543132 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706336bc-8e5c-4519-a709-2c7870b9673e service neutron] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:56.553184 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706336bc-8e5c-4519-a709-2c7870b9673e service neutron] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:56.609335 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db7fa36-c727-4094-a432-4d1a913f7c4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:56.610439 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db7fa36-c727-4094-a432-4d1a913f7c4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:56.610747 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db7fa36-c727-4094-a432-4d1a913f7c4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:56.611012 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db7fa36-c727-4094-a432-4d1a913f7c4f admin admin] PATH_INFO: `/v3/users/2fada062ec9a405986a35df1e5d6d356` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:56.612195 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db7fa36-c727-4094-a432-4d1a913f7c4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2fada062ec9a405986a35df1e5d6d356)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:56.614127 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db7fa36-c727-4094-a432-4d1a913f7c4f admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:56.666107 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-3db7fa36-c727-4094-a432-4d1a913f7c4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2fada062ec9a405986a35df1e5d6d356'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:56.675694 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-3db7fa36-c727-4094-a432-4d1a913f7c4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2fada062ec9a405986a35df1e5d6d356'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:56.676342 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-3db7fa36-c727-4094-a432-4d1a913f7c4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2fada062ec9a405986a35df1e5d6d356'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:56.676949 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-3db7fa36-c727-4094-a432-4d1a913f7c4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2fada062ec9a405986a35df1e5d6d356'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:56.684999 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:40:56.691755 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-3db7fa36-c727-4094-a432-4d1a913f7c4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2fada062ec9a405986a35df1e5d6d356'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:56.705996 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1231/2453] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:40:56 2022] DELETE /identity/v3/users/2fada062ec9a405986a35df1e5d6d356 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:40:56.983352 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6ed103-9e24-4778-8f8e-a3ce83510763 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:56.985847 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-bd6ed103-9e24-4778-8f8e-a3ce83510763 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:57.005084 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6ed103-9e24-4778-8f8e-a3ce83510763 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:57.067760 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aada0d84-ba19-44c4-b13b-6938c78effb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:57.068391 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aada0d84-ba19-44c4-b13b-6938c78effb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:57.068670 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aada0d84-ba19-44c4-b13b-6938c78effb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:57.068941 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aada0d84-ba19-44c4-b13b-6938c78effb5 admin admin] PATH_INFO: `/v3/projects/710e09d9dd3044b9b65d214269cbc1e5` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:57.070042 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aada0d84-ba19-44c4-b13b-6938c78effb5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=710e09d9dd3044b9b65d214269cbc1e5)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:57.071952 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aada0d84-ba19-44c4-b13b-6938c78effb5 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:57.118968 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-aada0d84-ba19-44c4-b13b-6938c78effb5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '710e09d9dd3044b9b65d214269cbc1e5', 'request_id': 'req-aada0d84-ba19-44c4-b13b-6938c78effb5'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:57.125953 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-aada0d84-ba19-44c4-b13b-6938c78effb5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '710e09d9dd3044b9b65d214269cbc1e5', 'request_id': 'req-aada0d84-ba19-44c4-b13b-6938c78effb5'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:57.134129 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-aada0d84-ba19-44c4-b13b-6938c78effb5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '710e09d9dd3044b9b65d214269cbc1e5', 'request_id': 'req-aada0d84-ba19-44c4-b13b-6938c78effb5'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:57.142260 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:40:57.147904 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-aada0d84-ba19-44c4-b13b-6938c78effb5 admin admin] The token cache is being invalidate because project 710e09d9dd3044b9b65d214269cbc1e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:40:57.148179 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-aada0d84-ba19-44c4-b13b-6938c78effb5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:57.149052 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-aada0d84-ba19-44c4-b13b-6938c78effb5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:40:57.154009 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1223/2454] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:40:56 2022] DELETE /identity/v3/projects/710e09d9dd3044b9b65d214269cbc1e5 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:40:57.159059 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac18cdec-420d-4efd-97cc-c278c4ff999b None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:57.159548 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac18cdec-420d-4efd-97cc-c278c4ff999b None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:57.159772 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac18cdec-420d-4efd-97cc-c278c4ff999b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:57.180249 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-ac18cdec-420d-4efd-97cc-c278c4ff999b None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:40:57.181434 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-ac18cdec-420d-4efd-97cc-c278c4ff999b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:57.239956 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1232/2455] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:40:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:40:57.247672 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fd962e-9602-498d-976a-2a4382d82c48 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:57.247953 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fd962e-9602-498d-976a-2a4382d82c48 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:57.248223 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fd962e-9602-498d-976a-2a4382d82c48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:57.268063 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-b5fd962e-9602-498d-976a-2a4382d82c48 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:40:57.269240 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-b5fd962e-9602-498d-976a-2a4382d82c48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:57.277394 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1224/2456] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:40:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:40:57.283480 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac18cdec-420d-4efd-97cc-c278c4ff999b None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:57.293084 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac18cdec-420d-4efd-97cc-c278c4ff999b None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:57.298327 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d1354f-ea0f-4298-913b-e6409094e349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:57.298920 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d1354f-ea0f-4298-913b-e6409094e349 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:57.299217 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d1354f-ea0f-4298-913b-e6409094e349 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:57.299487 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d1354f-ea0f-4298-913b-e6409094e349 admin admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:57.300115 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d1354f-ea0f-4298-913b-e6409094e349 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:40:57.300411 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d1354f-ea0f-4298-913b-e6409094e349 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:57.301980 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d1354f-ea0f-4298-913b-e6409094e349 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:57.313293 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1233/2457] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Oct 31 12:40:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:40:57.317454 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fd962e-9602-498d-976a-2a4382d82c48 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:57.327245 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fd962e-9602-498d-976a-2a4382d82c48 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:57.332589 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00d52f3-3fa0-42b7-9689-356f76d1e15d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:57.333190 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00d52f3-3fa0-42b7-9689-356f76d1e15d admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:57.333477 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00d52f3-3fa0-42b7-9689-356f76d1e15d admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:57.333863 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00d52f3-3fa0-42b7-9689-356f76d1e15d admin admin] PATH_INFO: `/v3/projects` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:57.334507 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00d52f3-3fa0-42b7-9689-356f76d1e15d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:57.336409 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00d52f3-3fa0-42b7-9689-356f76d1e15d admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:57.364955 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1225/2458] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:40:57 2022] POST /identity/v3/projects => generated 365 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:40:57.368876 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d1354f-ea0f-4298-913b-e6409094e349 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:57.379176 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d1354f-ea0f-4298-913b-e6409094e349 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:57.434789 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e945efe7-ab12-4f95-a36f-77b259626de6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:57.435427 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e945efe7-ab12-4f95-a36f-77b259626de6 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:57.435709 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e945efe7-ab12-4f95-a36f-77b259626de6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:57.435981 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e945efe7-ab12-4f95-a36f-77b259626de6 admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:57.436602 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e945efe7-ab12-4f95-a36f-77b259626de6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:57.438962 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e945efe7-ab12-4f95-a36f-77b259626de6 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:57.460471 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1234/2459] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:40:57 2022] POST /identity/v3/users => generated 283 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:40:57.464356 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00d52f3-3fa0-42b7-9689-356f76d1e15d admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:57.474785 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00d52f3-3fa0-42b7-9689-356f76d1e15d admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:57.479302 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab9bdb9-342e-4f3f-a188-85e2187f1c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:57.479889 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab9bdb9-342e-4f3f-a188-85e2187f1c85 admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:57.480162 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab9bdb9-342e-4f3f-a188-85e2187f1c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:57.480429 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab9bdb9-342e-4f3f-a188-85e2187f1c85 admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:57.481054 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab9bdb9-342e-4f3f-a188-85e2187f1c85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:57.482951 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab9bdb9-342e-4f3f-a188-85e2187f1c85 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:57.493551 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1226/2460] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:40:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 31 12:40:57.497208 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e945efe7-ab12-4f95-a36f-77b259626de6 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:57.507932 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e945efe7-ab12-4f95-a36f-77b259626de6 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:57.512787 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a371ff-0056-4f2e-b8e1-d7ad60b108cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:57.513378 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a371ff-0056-4f2e-b8e1-d7ad60b108cf admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:57.513653 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a371ff-0056-4f2e-b8e1-d7ad60b108cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:57.513922 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a371ff-0056-4f2e-b8e1-d7ad60b108cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:57.514547 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a371ff-0056-4f2e-b8e1-d7ad60b108cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:57.516398 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a371ff-0056-4f2e-b8e1-d7ad60b108cf admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:57.527079 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1235/2461] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:40:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:40:57.530996 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab9bdb9-342e-4f3f-a188-85e2187f1c85 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:57.541078 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab9bdb9-342e-4f3f-a188-85e2187f1c85 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:57.545507 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8728ec14-923e-40b5-8e0b-e30913f7e0f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:57.546148 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8728ec14-923e-40b5-8e0b-e30913f7e0f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:57.546422 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8728ec14-923e-40b5-8e0b-e30913f7e0f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:57.546691 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8728ec14-923e-40b5-8e0b-e30913f7e0f1 admin admin] PATH_INFO: `/v3/projects/aefe903d064d4ca8bc8f8db5c7cfd265/users/8efaa83c36c049e5a0b58b16e256cfdf/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:57.561459 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8728ec14-923e-40b5-8e0b-e30913f7e0f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aefe903d064d4ca8bc8f8db5c7cfd265, user_id=8efaa83c36c049e5a0b58b16e256cfdf, role_id=ca1a5695f8bd499aa0b8cedaeecd2cc6)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:57.564984 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8728ec14-923e-40b5-8e0b-e30913f7e0f1 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:57.577393 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1227/2462] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:40:57 2022] PUT /identity/v3/projects/aefe903d064d4ca8bc8f8db5c7cfd265/users/8efaa83c36c049e5a0b58b16e256cfdf/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 31 12:40:57.589147 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0bf664-1026-41bb-9e71-0a3312abbff6 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:57.589423 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0bf664-1026-41bb-9e71-0a3312abbff6 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:57.589690 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0bf664-1026-41bb-9e71-0a3312abbff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:57.617346 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-3e0bf664-1026-41bb-9e71-0a3312abbff6 None None] MFA Rules not processed for user `8efaa83c36c049e5a0b58b16e256cfdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:40:57.618465 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-3e0bf664-1026-41bb-9e71-0a3312abbff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:57.651685 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:40:57.685762 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1236/2463] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:40:57 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:40:57.691551 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8728ec14-923e-40b5-8e0b-e30913f7e0f1 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:57.752044 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8728ec14-923e-40b5-8e0b-e30913f7e0f1 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:57.756623 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8446ec-0247-449c-b5aa-ab2af9ef6a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:57.757213 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8446ec-0247-449c-b5aa-ab2af9ef6a00 admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:57.757489 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8446ec-0247-449c-b5aa-ab2af9ef6a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:57.757759 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8446ec-0247-449c-b5aa-ab2af9ef6a00 admin admin] PATH_INFO: `/v3/projects` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:57.758400 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8446ec-0247-449c-b5aa-ab2af9ef6a00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:57.760311 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8446ec-0247-449c-b5aa-ab2af9ef6a00 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:57.786327 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1228/2464] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:40:57 2022] POST /identity/v3/projects => generated 363 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 31 12:40:57.790511 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0bf664-1026-41bb-9e71-0a3312abbff6 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:57.802745 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0bf664-1026-41bb-9e71-0a3312abbff6 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:57.859242 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c0dd75-abd4-4bc6-bd85-f894d6505328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:57.860418 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c0dd75-abd4-4bc6-bd85-f894d6505328 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:57.860691 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c0dd75-abd4-4bc6-bd85-f894d6505328 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:57.860986 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c0dd75-abd4-4bc6-bd85-f894d6505328 admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:57.861611 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c0dd75-abd4-4bc6-bd85-f894d6505328 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:57.863547 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c0dd75-abd4-4bc6-bd85-f894d6505328 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:57.883753 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1237/2465] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:40:57 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:40:57.887848 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8446ec-0247-449c-b5aa-ab2af9ef6a00 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:57.898455 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8446ec-0247-449c-b5aa-ab2af9ef6a00 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:57.902971 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f985ca63-440e-44a6-9244-132063ad0e14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:57.903585 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f985ca63-440e-44a6-9244-132063ad0e14 admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:57.903859 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f985ca63-440e-44a6-9244-132063ad0e14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:57.904123 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f985ca63-440e-44a6-9244-132063ad0e14 admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:57.904753 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f985ca63-440e-44a6-9244-132063ad0e14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:57.906970 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f985ca63-440e-44a6-9244-132063ad0e14 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:57.917328 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1229/2466] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:40:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:40:57.921358 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c0dd75-abd4-4bc6-bd85-f894d6505328 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:57.931667 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c0dd75-abd4-4bc6-bd85-f894d6505328 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:57.936170 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416bf68f-ace1-4601-9d80-52f3fa7d9195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:57.936802 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416bf68f-ace1-4601-9d80-52f3fa7d9195 admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:57.937074 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416bf68f-ace1-4601-9d80-52f3fa7d9195 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:57.937341 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416bf68f-ace1-4601-9d80-52f3fa7d9195 admin admin] PATH_INFO: `/v3/projects/5f019e8c8e2540c685f605bd1c2670b8/users/a4de676f54d74727a9c05a9739589ef7/roles/176c07b4fe3b4537bdfeb4be21b379c5` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:57.951517 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416bf68f-ace1-4601-9d80-52f3fa7d9195 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f019e8c8e2540c685f605bd1c2670b8, user_id=a4de676f54d74727a9c05a9739589ef7, role_id=176c07b4fe3b4537bdfeb4be21b379c5)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:57.954259 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416bf68f-ace1-4601-9d80-52f3fa7d9195 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:57.966834 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1238/2467] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:40:57 2022] PUT /identity/v3/projects/5f019e8c8e2540c685f605bd1c2670b8/users/a4de676f54d74727a9c05a9739589ef7/roles/176c07b4fe3b4537bdfeb4be21b379c5 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 31 12:40:57.971414 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f985ca63-440e-44a6-9244-132063ad0e14 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:57.981230 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f985ca63-440e-44a6-9244-132063ad0e14 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:58.041213 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800e7f9c-55f4-4533-b0c4-7e147124b1d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:58.041858 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800e7f9c-55f4-4533-b0c4-7e147124b1d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:58.042133 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800e7f9c-55f4-4533-b0c4-7e147124b1d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:58.042431 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800e7f9c-55f4-4533-b0c4-7e147124b1d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:58.043454 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800e7f9c-55f4-4533-b0c4-7e147124b1d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:58.045425 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800e7f9c-55f4-4533-b0c4-7e147124b1d3 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:58.057119 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1230/2468] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:40:57 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:40:58.061422 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416bf68f-ace1-4601-9d80-52f3fa7d9195 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:58.072010 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416bf68f-ace1-4601-9d80-52f3fa7d9195 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:58.076490 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0316d9-8a16-479b-9f38-e070fcf81369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:58.077135 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0316d9-8a16-479b-9f38-e070fcf81369 admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:58.077410 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0316d9-8a16-479b-9f38-e070fcf81369 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:58.077678 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0316d9-8a16-479b-9f38-e070fcf81369 admin admin] PATH_INFO: `/v3/projects/5f019e8c8e2540c685f605bd1c2670b8/users/a4de676f54d74727a9c05a9739589ef7/roles/176c07b4fe3b4537bdfeb4be21b379c5` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:58.079684 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0316d9-8a16-479b-9f38-e070fcf81369 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f019e8c8e2540c685f605bd1c2670b8, user_id=a4de676f54d74727a9c05a9739589ef7, role_id=176c07b4fe3b4537bdfeb4be21b379c5)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:58.082270 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0316d9-8a16-479b-9f38-e070fcf81369 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:58.092557 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1239/2469] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:40:58 2022] PUT /identity/v3/projects/5f019e8c8e2540c685f605bd1c2670b8/users/a4de676f54d74727a9c05a9739589ef7/roles/176c07b4fe3b4537bdfeb4be21b379c5 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:40:58.103207 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c48a6e5-ec35-49cb-b240-087be1461dd8 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:58.103502 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c48a6e5-ec35-49cb-b240-087be1461dd8 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:58.103781 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c48a6e5-ec35-49cb-b240-087be1461dd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:58.112835 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:40:58.132951 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-8c48a6e5-ec35-49cb-b240-087be1461dd8 None None] MFA Rules not processed for user `a4de676f54d74727a9c05a9739589ef7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:40:58.134197 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-8c48a6e5-ec35-49cb-b240-087be1461dd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:58.208866 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1231/2470] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:40:58 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:40:58.216210 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0316d9-8a16-479b-9f38-e070fcf81369 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:58.217927 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-3e0316d9-8a16-479b-9f38-e070fcf81369 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:40:58.277557 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0316d9-8a16-479b-9f38-e070fcf81369 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:58.281853 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5824ac0-ceb1-4523-bd05-c1a6b7b84efa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d6d7ea077d49e8a66c6c6762499702', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:58.282465 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5824ac0-ceb1-4523-bd05-c1a6b7b84efa service cinder] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:58.282742 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5824ac0-ceb1-4523-bd05-c1a6b7b84efa service cinder] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:58.283025 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5824ac0-ceb1-4523-bd05-c1a6b7b84efa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:58.291851 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5824ac0-ceb1-4523-bd05-c1a6b7b84efa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:58.293264 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5824ac0-ceb1-4523-bd05-c1a6b7b84efa service cinder] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:58.303105 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1240/2471] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:40:58 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 31 12:40:58.379296 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c48a6e5-ec35-49cb-b240-087be1461dd8 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:40:58.389105 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c48a6e5-ec35-49cb-b240-087be1461dd8 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:40:58.393014 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f067ccd-5b02-4f48-8828-4017bdafe5de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d6d7ea077d49e8a66c6c6762499702', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:40:58.393608 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f067ccd-5b02-4f48-8828-4017bdafe5de service cinder] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:40:58.393884 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f067ccd-5b02-4f48-8828-4017bdafe5de service cinder] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:40:58.394149 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f067ccd-5b02-4f48-8828-4017bdafe5de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:40:58.450183 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f067ccd-5b02-4f48-8828-4017bdafe5de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:40:58.451638 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f067ccd-5b02-4f48-8828-4017bdafe5de service cinder] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:40:58.460967 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1232/2472] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:40:58 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:41:01.179238 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5824ac0-ceb1-4523-bd05-c1a6b7b84efa service cinder] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:41:01.212370 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:41:01.244722 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5824ac0-ceb1-4523-bd05-c1a6b7b84efa service cinder] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:41:01.249491 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c2a5f6-43e6-470d-bf60-1da380650500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:41:01.250097 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c2a5f6-43e6-470d-bf60-1da380650500 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:01.250371 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c2a5f6-43e6-470d-bf60-1da380650500 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:01.250637 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c2a5f6-43e6-470d-bf60-1da380650500 admin admin] PATH_INFO: `/v3/users/8efaa83c36c049e5a0b58b16e256cfdf` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:01.251796 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c2a5f6-43e6-470d-bf60-1da380650500 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8efaa83c36c049e5a0b58b16e256cfdf)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:41:01.253771 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c2a5f6-43e6-470d-bf60-1da380650500 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:41:01.302629 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-96c2a5f6-43e6-470d-bf60-1da380650500 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8efaa83c36c049e5a0b58b16e256cfdf'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:41:01.310345 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-96c2a5f6-43e6-470d-bf60-1da380650500 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8efaa83c36c049e5a0b58b16e256cfdf'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:41:01.311203 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-96c2a5f6-43e6-470d-bf60-1da380650500 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8efaa83c36c049e5a0b58b16e256cfdf'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:41:01.311901 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-96c2a5f6-43e6-470d-bf60-1da380650500 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8efaa83c36c049e5a0b58b16e256cfdf'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:41:01.327311 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-96c2a5f6-43e6-470d-bf60-1da380650500 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8efaa83c36c049e5a0b58b16e256cfdf'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:41:01.347238 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1241/2473] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:41:01 2022] DELETE /identity/v3/users/8efaa83c36c049e5a0b58b16e256cfdf => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:41:01.351718 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8d5364-3d33-41e3-8b09-adb12bd52c65 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:01.352200 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8d5364-3d33-41e3-8b09-adb12bd52c65 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:01.352300 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8d5364-3d33-41e3-8b09-adb12bd52c65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:01.375426 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-9b8d5364-3d33-41e3-8b09-adb12bd52c65 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:41:01.376646 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-9b8d5364-3d33-41e3-8b09-adb12bd52c65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:41:01.438555 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1233/2474] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:41:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 31 12:41:01.448477 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c2a5f6-43e6-470d-bf60-1da380650500 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:41:01.449946 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-96c2a5f6-43e6-470d-bf60-1da380650500 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:41:01.509906 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c2a5f6-43e6-470d-bf60-1da380650500 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:41:01.513874 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f25c3ee-8ad2-4f2a-a210-888127ef547d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:41:01.514476 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f25c3ee-8ad2-4f2a-a210-888127ef547d service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:01.514753 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f25c3ee-8ad2-4f2a-a210-888127ef547d service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:01.515020 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f25c3ee-8ad2-4f2a-a210-888127ef547d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:01.525543 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f25c3ee-8ad2-4f2a-a210-888127ef547d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:41:01.526959 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f25c3ee-8ad2-4f2a-a210-888127ef547d service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:41:01.536950 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1242/2475] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:41:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:41:01.824063 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8d5364-3d33-41e3-8b09-adb12bd52c65 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:41:01.825865 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-9b8d5364-3d33-41e3-8b09-adb12bd52c65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:41:01.838220 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8d5364-3d33-41e3-8b09-adb12bd52c65 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:41:01.844847 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c60230f-d591-42e4-8adb-065081883ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:41:01.845516 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c60230f-d591-42e4-8adb-065081883ea7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:01.845792 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c60230f-d591-42e4-8adb-065081883ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:01.846092 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c60230f-d591-42e4-8adb-065081883ea7 admin admin] PATH_INFO: `/v3/projects/aefe903d064d4ca8bc8f8db5c7cfd265` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:01.847265 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c60230f-d591-42e4-8adb-065081883ea7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aefe903d064d4ca8bc8f8db5c7cfd265)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:41:01.849097 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c60230f-d591-42e4-8adb-065081883ea7 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:41:01.877925 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:41:01.907023 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-4c60230f-d591-42e4-8adb-065081883ea7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aefe903d064d4ca8bc8f8db5c7cfd265', 'request_id': 'req-4c60230f-d591-42e4-8adb-065081883ea7'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:41:01.913233 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-4c60230f-d591-42e4-8adb-065081883ea7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aefe903d064d4ca8bc8f8db5c7cfd265', 'request_id': 'req-4c60230f-d591-42e4-8adb-065081883ea7'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:41:01.921019 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-4c60230f-d591-42e4-8adb-065081883ea7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aefe903d064d4ca8bc8f8db5c7cfd265', 'request_id': 'req-4c60230f-d591-42e4-8adb-065081883ea7'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:41:01.933816 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-4c60230f-d591-42e4-8adb-065081883ea7 admin admin] The token cache is being invalidate because project aefe903d064d4ca8bc8f8db5c7cfd265 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:41:01.934099 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-4c60230f-d591-42e4-8adb-065081883ea7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:41:01.934909 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-4c60230f-d591-42e4-8adb-065081883ea7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:41:01.936633 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1234/2476] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:41:01 2022] DELETE /identity/v3/projects/aefe903d064d4ca8bc8f8db5c7cfd265 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:41:01.940799 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f25c3ee-8ad2-4f2a-a210-888127ef547d service neutron] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:41:01.951383 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f25c3ee-8ad2-4f2a-a210-888127ef547d service neutron] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:41:02.000205 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:41:02.011520 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaf5781-6244-4ba6-b852-6fa0b1d976eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:41:02.012172 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaf5781-6244-4ba6-b852-6fa0b1d976eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:02.012509 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaf5781-6244-4ba6-b852-6fa0b1d976eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:02.012844 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaf5781-6244-4ba6-b852-6fa0b1d976eb admin admin] PATH_INFO: `/v3/users/a4de676f54d74727a9c05a9739589ef7` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:02.014051 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaf5781-6244-4ba6-b852-6fa0b1d976eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4de676f54d74727a9c05a9739589ef7)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:41:02.016594 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaf5781-6244-4ba6-b852-6fa0b1d976eb admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:41:02.064411 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-eaaf5781-6244-4ba6-b852-6fa0b1d976eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4de676f54d74727a9c05a9739589ef7'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:41:02.072125 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-eaaf5781-6244-4ba6-b852-6fa0b1d976eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4de676f54d74727a9c05a9739589ef7'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:41:02.072841 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-eaaf5781-6244-4ba6-b852-6fa0b1d976eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4de676f54d74727a9c05a9739589ef7'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:41:02.073839 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-eaaf5781-6244-4ba6-b852-6fa0b1d976eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4de676f54d74727a9c05a9739589ef7'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:41:02.088289 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-eaaf5781-6244-4ba6-b852-6fa0b1d976eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4de676f54d74727a9c05a9739589ef7'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:41:02.100674 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1243/2477] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:41:01 2022] DELETE /identity/v3/users/a4de676f54d74727a9c05a9739589ef7 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 31 12:41:02.377038 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c60230f-d591-42e4-8adb-065081883ea7 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:41:02.378464 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-4c60230f-d591-42e4-8adb-065081883ea7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:41:02.393960 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c60230f-d591-42e4-8adb-065081883ea7 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:41:02.459130 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bdf3f7-fe47-4fc6-ae18-f1a69f81b9f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:41:02.459822 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bdf3f7-fe47-4fc6-ae18-f1a69f81b9f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:02.460152 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bdf3f7-fe47-4fc6-ae18-f1a69f81b9f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:02.460451 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bdf3f7-fe47-4fc6-ae18-f1a69f81b9f4 admin admin] PATH_INFO: `/v3/projects/5f019e8c8e2540c685f605bd1c2670b8` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:02.461582 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bdf3f7-fe47-4fc6-ae18-f1a69f81b9f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f019e8c8e2540c685f605bd1c2670b8)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:41:02.463511 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bdf3f7-fe47-4fc6-ae18-f1a69f81b9f4 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:41:02.512445 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-77bdf3f7-fe47-4fc6-ae18-f1a69f81b9f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f019e8c8e2540c685f605bd1c2670b8', 'request_id': 'req-77bdf3f7-fe47-4fc6-ae18-f1a69f81b9f4'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:41:02.518390 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-77bdf3f7-fe47-4fc6-ae18-f1a69f81b9f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f019e8c8e2540c685f605bd1c2670b8', 'request_id': 'req-77bdf3f7-fe47-4fc6-ae18-f1a69f81b9f4'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:41:02.526510 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-77bdf3f7-fe47-4fc6-ae18-f1a69f81b9f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f019e8c8e2540c685f605bd1c2670b8', 'request_id': 'req-77bdf3f7-fe47-4fc6-ae18-f1a69f81b9f4'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:41:02.538585 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-77bdf3f7-fe47-4fc6-ae18-f1a69f81b9f4 admin admin] The token cache is being invalidate because project 5f019e8c8e2540c685f605bd1c2670b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:41:02.539013 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-77bdf3f7-fe47-4fc6-ae18-f1a69f81b9f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:41:02.539751 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-77bdf3f7-fe47-4fc6-ae18-f1a69f81b9f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:41:02.541437 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1235/2478] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:41:02 2022] DELETE /identity/v3/projects/5f019e8c8e2540c685f605bd1c2670b8 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:41:02.546509 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8329f8fb-544d-4886-a177-9adcc2baa11f None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:02.548249 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8329f8fb-544d-4886-a177-9adcc2baa11f None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:02.548249 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8329f8fb-544d-4886-a177-9adcc2baa11f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:02.570129 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-8329f8fb-544d-4886-a177-9adcc2baa11f None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:41:02.570846 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-8329f8fb-544d-4886-a177-9adcc2baa11f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:41:02.634827 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1244/2479] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:41:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:41:02.642910 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a29f083-3cf9-4bb5-bece-b3953d4a0f1a None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:02.643313 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a29f083-3cf9-4bb5-bece-b3953d4a0f1a None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:02.643652 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a29f083-3cf9-4bb5-bece-b3953d4a0f1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:02.664105 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-4a29f083-3cf9-4bb5-bece-b3953d4a0f1a None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:41:02.665333 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-4a29f083-3cf9-4bb5-bece-b3953d4a0f1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:41:02.673917 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1236/2480] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:41:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 31 12:41:02.678140 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8329f8fb-544d-4886-a177-9adcc2baa11f None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:41:02.687909 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8329f8fb-544d-4886-a177-9adcc2baa11f None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:41:02.692602 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9f1773-257a-43e0-811c-cfbc7a8882da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:41:02.693202 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9f1773-257a-43e0-811c-cfbc7a8882da admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:02.693477 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9f1773-257a-43e0-811c-cfbc7a8882da admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:02.693746 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9f1773-257a-43e0-811c-cfbc7a8882da admin admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:02.694412 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9f1773-257a-43e0-811c-cfbc7a8882da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:41:02.694710 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9f1773-257a-43e0-811c-cfbc7a8882da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:41:02.696316 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9f1773-257a-43e0-811c-cfbc7a8882da admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:41:02.708468 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1245/2481] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Oct 31 12:41:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:41:02.712574 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a29f083-3cf9-4bb5-bece-b3953d4a0f1a None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:41:02.723047 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a29f083-3cf9-4bb5-bece-b3953d4a0f1a None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:41:02.727706 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6e6a63-e527-4692-b130-92f2de5a046c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:41:02.728301 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6e6a63-e527-4692-b130-92f2de5a046c admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:02.728580 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6e6a63-e527-4692-b130-92f2de5a046c admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:02.728851 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6e6a63-e527-4692-b130-92f2de5a046c admin admin] PATH_INFO: `/v3/projects` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:02.729475 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6e6a63-e527-4692-b130-92f2de5a046c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:41:02.732083 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6e6a63-e527-4692-b130-92f2de5a046c admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:41:02.761579 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1237/2482] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:41:02 2022] POST /identity/v3/projects => generated 377 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:41:02.766112 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9f1773-257a-43e0-811c-cfbc7a8882da admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:41:02.776168 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9f1773-257a-43e0-811c-cfbc7a8882da admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:41:02.833123 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bac714c-b290-4844-95cd-92047fd292c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:41:02.833679 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bac714c-b290-4844-95cd-92047fd292c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:02.834042 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bac714c-b290-4844-95cd-92047fd292c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:02.834230 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bac714c-b290-4844-95cd-92047fd292c6 admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:02.834856 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bac714c-b290-4844-95cd-92047fd292c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:41:02.836838 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bac714c-b290-4844-95cd-92047fd292c6 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:41:02.845058 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:41:02.859990 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1246/2483] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:41:02 2022] POST /identity/v3/users => generated 289 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:41:02.864066 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6e6a63-e527-4692-b130-92f2de5a046c admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:41:02.874611 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6e6a63-e527-4692-b130-92f2de5a046c admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:41:02.880308 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c130d6-e437-4659-a001-911dfce17f8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:41:02.880902 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c130d6-e437-4659-a001-911dfce17f8d admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:02.881188 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c130d6-e437-4659-a001-911dfce17f8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:02.881459 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c130d6-e437-4659-a001-911dfce17f8d admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:02.882095 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c130d6-e437-4659-a001-911dfce17f8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:41:02.883709 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c130d6-e437-4659-a001-911dfce17f8d admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:41:02.893984 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1238/2484] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:41:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:41:02.897808 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bac714c-b290-4844-95cd-92047fd292c6 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:41:02.907713 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bac714c-b290-4844-95cd-92047fd292c6 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:41:02.912229 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4075b387-d8f5-477a-bf49-34ea554bade9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:41:02.912815 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4075b387-d8f5-477a-bf49-34ea554bade9 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:02.913090 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4075b387-d8f5-477a-bf49-34ea554bade9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:02.913362 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4075b387-d8f5-477a-bf49-34ea554bade9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:02.913985 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4075b387-d8f5-477a-bf49-34ea554bade9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:41:02.915617 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4075b387-d8f5-477a-bf49-34ea554bade9 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:41:02.926718 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1247/2485] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:41:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:41:02.931000 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c130d6-e437-4659-a001-911dfce17f8d admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:41:02.937273 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:41:02.944442 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c130d6-e437-4659-a001-911dfce17f8d admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:41:02.950504 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9903e401-733d-4260-bd91-fbe051753e50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:41:02.951145 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9903e401-733d-4260-bd91-fbe051753e50 admin admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:02.951451 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9903e401-733d-4260-bd91-fbe051753e50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:02.951720 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9903e401-733d-4260-bd91-fbe051753e50 admin admin] PATH_INFO: `/v3/projects/871c528fb3a24f17a3ea12bce679cffe/users/fdf8ee32608a492d91528bec83e86d94/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:02.965966 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9903e401-733d-4260-bd91-fbe051753e50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=871c528fb3a24f17a3ea12bce679cffe, user_id=fdf8ee32608a492d91528bec83e86d94, role_id=ca1a5695f8bd499aa0b8cedaeecd2cc6)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:41:02.968717 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9903e401-733d-4260-bd91-fbe051753e50 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:41:02.981983 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1239/2486] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:41:02 2022] PUT /identity/v3/projects/871c528fb3a24f17a3ea12bce679cffe/users/fdf8ee32608a492d91528bec83e86d94/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:41:02.993597 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb0514c-6949-4837-b84a-1ce20cdbe857 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:02.993887 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb0514c-6949-4837-b84a-1ce20cdbe857 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:02.994160 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb0514c-6949-4837-b84a-1ce20cdbe857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:03.029096 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-6fb0514c-6949-4837-b84a-1ce20cdbe857 None None] MFA Rules not processed for user `fdf8ee32608a492d91528bec83e86d94`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:41:03.030692 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-6fb0514c-6949-4837-b84a-1ce20cdbe857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:41:03.071125 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9903e401-733d-4260-bd91-fbe051753e50 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:41:03.072948 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-9903e401-733d-4260-bd91-fbe051753e50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:41:03.098076 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1248/2487] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:41:02 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:41:03.102346 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb0514c-6949-4837-b84a-1ce20cdbe857 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:41:03.164503 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb0514c-6949-4837-b84a-1ce20cdbe857 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:41:03.165456 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9903e401-733d-4260-bd91-fbe051753e50 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:41:03.169103 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503478ca-6f56-42f6-8493-c7631fff062a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:41:03.169711 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503478ca-6f56-42f6-8493-c7631fff062a admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:03.169993 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503478ca-6f56-42f6-8493-c7631fff062a admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:03.170262 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503478ca-6f56-42f6-8493-c7631fff062a admin admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:03.170554 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a080fe6-fa1c-4451-80fc-b1e23a66f055 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e7777495d7a406c8c50b1ad52e47263', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:41:03.170910 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503478ca-6f56-42f6-8493-c7631fff062a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:41:03.171187 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a080fe6-fa1c-4451-80fc-b1e23a66f055 service placement] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:03.171457 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a080fe6-fa1c-4451-80fc-b1e23a66f055 service placement] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:03.171839 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a080fe6-fa1c-4451-80fc-b1e23a66f055 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:03.173198 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503478ca-6f56-42f6-8493-c7631fff062a admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:41:03.173592 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-6a080fe6-fa1c-4451-80fc-b1e23a66f055 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:41:03.185466 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a080fe6-fa1c-4451-80fc-b1e23a66f055 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:41:03.187023 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a080fe6-fa1c-4451-80fc-b1e23a66f055 service placement] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:41:03.198744 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1240/2488] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:41:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 31 12:41:03.204344 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1249/2489] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:41:03 2022] POST /identity/v3/projects => generated 377 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 31 12:41:03.205434 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a080fe6-fa1c-4451-80fc-b1e23a66f055 service placement] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:41:03.215370 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a080fe6-fa1c-4451-80fc-b1e23a66f055 service placement] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:41:03.276193 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e453b2ce-ba0d-43e7-af95-12b4e752bba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:41:03.276804 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e453b2ce-ba0d-43e7-af95-12b4e752bba1 admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:03.277085 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e453b2ce-ba0d-43e7-af95-12b4e752bba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:03.277357 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e453b2ce-ba0d-43e7-af95-12b4e752bba1 admin admin] PATH_INFO: `/v3/users` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:03.277978 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e453b2ce-ba0d-43e7-af95-12b4e752bba1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:41:03.279912 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e453b2ce-ba0d-43e7-af95-12b4e752bba1 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:41:03.302996 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1241/2490] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:41:03 2022] POST /identity/v3/users => generated 295 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:41:03.306799 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503478ca-6f56-42f6-8493-c7631fff062a admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:41:03.318067 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503478ca-6f56-42f6-8493-c7631fff062a admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:41:03.322908 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d527e1-4266-4ea5-a1d7-6539c234aa7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:41:03.323658 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d527e1-4266-4ea5-a1d7-6539c234aa7b admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:03.323978 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d527e1-4266-4ea5-a1d7-6539c234aa7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:03.324252 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d527e1-4266-4ea5-a1d7-6539c234aa7b admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:03.324887 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d527e1-4266-4ea5-a1d7-6539c234aa7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:41:03.326460 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d527e1-4266-4ea5-a1d7-6539c234aa7b admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:41:03.340804 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1250/2491] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:41:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:41:03.341986 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e453b2ce-ba0d-43e7-af95-12b4e752bba1 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:41:03.352503 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e453b2ce-ba0d-43e7-af95-12b4e752bba1 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:41:03.357215 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d88b8c-ecca-4de0-8192-75032379b050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:41:03.357861 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d88b8c-ecca-4de0-8192-75032379b050 admin admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:03.358138 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d88b8c-ecca-4de0-8192-75032379b050 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:03.358407 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d88b8c-ecca-4de0-8192-75032379b050 admin admin] PATH_INFO: `/v3/projects/de13e8ffdfde4a07a19471ed7e5a5e1f/users/d9a2c5c4148941029b52713a30c39122/roles/176c07b4fe3b4537bdfeb4be21b379c5` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:03.373221 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d88b8c-ecca-4de0-8192-75032379b050 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de13e8ffdfde4a07a19471ed7e5a5e1f, user_id=d9a2c5c4148941029b52713a30c39122, role_id=176c07b4fe3b4537bdfeb4be21b379c5)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:41:03.376622 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d88b8c-ecca-4de0-8192-75032379b050 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:41:03.389999 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1242/2492] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:41:03 2022] PUT /identity/v3/projects/de13e8ffdfde4a07a19471ed7e5a5e1f/users/d9a2c5c4148941029b52713a30c39122/roles/176c07b4fe3b4537bdfeb4be21b379c5 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:41:03.395508 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d527e1-4266-4ea5-a1d7-6539c234aa7b admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:41:03.405762 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d527e1-4266-4ea5-a1d7-6539c234aa7b admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:41:03.448835 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:41:03.466860 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb9fc11-0c12-4c15-b7dd-fe1fb534a556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:41:03.467508 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb9fc11-0c12-4c15-b7dd-fe1fb534a556 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:03.467790 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb9fc11-0c12-4c15-b7dd-fe1fb534a556 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:03.468060 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb9fc11-0c12-4c15-b7dd-fe1fb534a556 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:03.468763 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb9fc11-0c12-4c15-b7dd-fe1fb534a556 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:41:03.470576 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb9fc11-0c12-4c15-b7dd-fe1fb534a556 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:41:03.480719 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1251/2493] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:41:03 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:41:03.485760 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d88b8c-ecca-4de0-8192-75032379b050 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:41:03.495972 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d88b8c-ecca-4de0-8192-75032379b050 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:41:03.501757 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58410095-70da-4854-8e81-bdde3fcd97a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:41:03.502403 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58410095-70da-4854-8e81-bdde3fcd97a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:03.502685 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58410095-70da-4854-8e81-bdde3fcd97a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:03.502962 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58410095-70da-4854-8e81-bdde3fcd97a5 admin admin] PATH_INFO: `/v3/projects/de13e8ffdfde4a07a19471ed7e5a5e1f/users/d9a2c5c4148941029b52713a30c39122/roles/176c07b4fe3b4537bdfeb4be21b379c5` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:03.504936 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58410095-70da-4854-8e81-bdde3fcd97a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de13e8ffdfde4a07a19471ed7e5a5e1f, user_id=d9a2c5c4148941029b52713a30c39122, role_id=176c07b4fe3b4537bdfeb4be21b379c5)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:41:03.507588 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58410095-70da-4854-8e81-bdde3fcd97a5 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:41:03.513863 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:41:03.519045 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1243/2494] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:41:03 2022] PUT /identity/v3/projects/de13e8ffdfde4a07a19471ed7e5a5e1f/users/d9a2c5c4148941029b52713a30c39122/roles/176c07b4fe3b4537bdfeb4be21b379c5 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:41:03.528862 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb25f84f-3270-4c1c-bbaa-e2786f6ad753 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:03.529147 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb25f84f-3270-4c1c-bbaa-e2786f6ad753 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:03.529418 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb25f84f-3270-4c1c-bbaa-e2786f6ad753 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:03.558756 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-cb25f84f-3270-4c1c-bbaa-e2786f6ad753 None None] MFA Rules not processed for user `d9a2c5c4148941029b52713a30c39122`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:41:03.559986 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-cb25f84f-3270-4c1c-bbaa-e2786f6ad753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:41:03.626960 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1252/2495] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:41:03 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:41:03.637409 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58410095-70da-4854-8e81-bdde3fcd97a5 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:41:03.639109 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-58410095-70da-4854-8e81-bdde3fcd97a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:41:03.699852 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58410095-70da-4854-8e81-bdde3fcd97a5 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:41:03.703628 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9912263-4e45-414e-a498-8a3c3e320081 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d6d7ea077d49e8a66c6c6762499702', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:41:03.704236 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9912263-4e45-414e-a498-8a3c3e320081 service cinder] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:03.704521 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9912263-4e45-414e-a498-8a3c3e320081 service cinder] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:03.704791 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9912263-4e45-414e-a498-8a3c3e320081 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:03.764433 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9912263-4e45-414e-a498-8a3c3e320081 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:41:03.765847 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9912263-4e45-414e-a498-8a3c3e320081 service cinder] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:41:03.774884 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1244/2496] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:41:03 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:41:10.550818 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb25f84f-3270-4c1c-bbaa-e2786f6ad753 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:41:10.560705 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb25f84f-3270-4c1c-bbaa-e2786f6ad753 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:41:10.564887 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58480171-ffb2-4d3d-bdca-2ea68868ae9a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d6d7ea077d49e8a66c6c6762499702', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:41:10.565490 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58480171-ffb2-4d3d-bdca-2ea68868ae9a service cinder] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:10.565775 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58480171-ffb2-4d3d-bdca-2ea68868ae9a service cinder] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:10.566046 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58480171-ffb2-4d3d-bdca-2ea68868ae9a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:10.567743 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-58480171-ffb2-4d3d-bdca-2ea68868ae9a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:41:10.627561 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58480171-ffb2-4d3d-bdca-2ea68868ae9a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:41:10.628983 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58480171-ffb2-4d3d-bdca-2ea68868ae9a service cinder] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:41:10.638110 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1253/2497] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:41:10 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:41:11.535541 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9912263-4e45-414e-a498-8a3c3e320081 service cinder] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:41:11.536921 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-d9912263-4e45-414e-a498-8a3c3e320081 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:41:11.570356 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9912263-4e45-414e-a498-8a3c3e320081 service cinder] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:41:11.573894 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7e35b6-0a28-4c8e-b51a-c3fdf8bfc2dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d6d7ea077d49e8a66c6c6762499702', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:41:11.574490 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7e35b6-0a28-4c8e-b51a-c3fdf8bfc2dd service cinder] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:11.574762 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7e35b6-0a28-4c8e-b51a-c3fdf8bfc2dd service cinder] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:11.575029 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7e35b6-0a28-4c8e-b51a-c3fdf8bfc2dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:11.584963 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7e35b6-0a28-4c8e-b51a-c3fdf8bfc2dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:41:11.586367 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7e35b6-0a28-4c8e-b51a-c3fdf8bfc2dd service cinder] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:41:11.595781 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1245/2498] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:41:11 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:41:20.634409 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=72 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:41:20.645718 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dad5cd-5e22-44a1-bf6c-251f29cf9af4 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:20.645994 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dad5cd-5e22-44a1-bf6c-251f29cf9af4 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:20.646260 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dad5cd-5e22-44a1-bf6c-251f29cf9af4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:20.665432 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-c6dad5cd-5e22-44a1-bf6c-251f29cf9af4 None None] MFA Rules not processed for user `49b76d81ee9446cf9014a2e695ef03e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:41:20.666625 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-c6dad5cd-5e22-44a1-bf6c-251f29cf9af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:41:20.674054 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1254/2499] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:41:20 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 31 12:41:21.347076 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7e35b6-0a28-4c8e-b51a-c3fdf8bfc2dd service cinder] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:41:21.348532 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-eb7e35b6-0a28-4c8e-b51a-c3fdf8bfc2dd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:41:21.414395 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7e35b6-0a28-4c8e-b51a-c3fdf8bfc2dd service cinder] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:41:21.418893 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d47674c-d890-4e4a-934c-8a5f5f20a727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:41:21.419553 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d47674c-d890-4e4a-934c-8a5f5f20a727 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:21.419815 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d47674c-d890-4e4a-934c-8a5f5f20a727 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:21.420080 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d47674c-d890-4e4a-934c-8a5f5f20a727 admin admin] PATH_INFO: `/v3/users/f0f42e37e20749738f302ff2defad3e8` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:21.421158 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d47674c-d890-4e4a-934c-8a5f5f20a727 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0f42e37e20749738f302ff2defad3e8)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:41:21.423095 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d47674c-d890-4e4a-934c-8a5f5f20a727 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:41:21.444538 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:41:21.468825 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-6d47674c-d890-4e4a-934c-8a5f5f20a727 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0f42e37e20749738f302ff2defad3e8'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:41:21.476719 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-6d47674c-d890-4e4a-934c-8a5f5f20a727 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0f42e37e20749738f302ff2defad3e8'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:41:21.477415 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-6d47674c-d890-4e4a-934c-8a5f5f20a727 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0f42e37e20749738f302ff2defad3e8'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:41:21.489435 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-6d47674c-d890-4e4a-934c-8a5f5f20a727 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0f42e37e20749738f302ff2defad3e8'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:41:21.496307 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-6d47674c-d890-4e4a-934c-8a5f5f20a727 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0f42e37e20749738f302ff2defad3e8'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:41:21.502376 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1246/2500] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:41:21 2022] DELETE /identity/v3/users/f0f42e37e20749738f302ff2defad3e8 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:41:21.506864 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5366b3a0-7345-4042-a68c-28f7eb8a5090 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:21.507147 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5366b3a0-7345-4042-a68c-28f7eb8a5090 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:21.507455 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5366b3a0-7345-4042-a68c-28f7eb8a5090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:21.526684 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-5366b3a0-7345-4042-a68c-28f7eb8a5090 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:41:21.527900 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-5366b3a0-7345-4042-a68c-28f7eb8a5090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:41:21.586906 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1255/2501] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:41:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:41:21.596727 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d47674c-d890-4e4a-934c-8a5f5f20a727 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:41:21.598268 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-6d47674c-d890-4e4a-934c-8a5f5f20a727 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:41:21.677135 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d47674c-d890-4e4a-934c-8a5f5f20a727 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:41:21.680820 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3313fa1f-3978-47ee-9d17-8d3619cf91f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:41:21.681419 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3313fa1f-3978-47ee-9d17-8d3619cf91f1 service neutron] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:21.681694 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3313fa1f-3978-47ee-9d17-8d3619cf91f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:21.681963 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3313fa1f-3978-47ee-9d17-8d3619cf91f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:21.690770 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3313fa1f-3978-47ee-9d17-8d3619cf91f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:41:21.692223 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3313fa1f-3978-47ee-9d17-8d3619cf91f1 service neutron] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:41:21.701475 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1247/2502] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:41:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:41:21.964177 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5366b3a0-7345-4042-a68c-28f7eb8a5090 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:41:21.965658 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-5366b3a0-7345-4042-a68c-28f7eb8a5090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:41:21.979014 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5366b3a0-7345-4042-a68c-28f7eb8a5090 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:41:21.984241 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2d3f91-7d5d-4cac-93d5-f59b15a809f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:41:21.986237 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2d3f91-7d5d-4cac-93d5-f59b15a809f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:21.986549 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2d3f91-7d5d-4cac-93d5-f59b15a809f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:21.986972 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2d3f91-7d5d-4cac-93d5-f59b15a809f9 admin admin] PATH_INFO: `/v3/projects/7947bfb1d9264cd9a226695ef8f8223c` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:21.988362 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2d3f91-7d5d-4cac-93d5-f59b15a809f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7947bfb1d9264cd9a226695ef8f8223c)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:41:21.990674 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2d3f91-7d5d-4cac-93d5-f59b15a809f9 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:41:22.040317 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-7b2d3f91-7d5d-4cac-93d5-f59b15a809f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7947bfb1d9264cd9a226695ef8f8223c', 'request_id': 'req-7b2d3f91-7d5d-4cac-93d5-f59b15a809f9'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:41:22.046901 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-7b2d3f91-7d5d-4cac-93d5-f59b15a809f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7947bfb1d9264cd9a226695ef8f8223c', 'request_id': 'req-7b2d3f91-7d5d-4cac-93d5-f59b15a809f9'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:41:22.056120 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-7b2d3f91-7d5d-4cac-93d5-f59b15a809f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7947bfb1d9264cd9a226695ef8f8223c', 'request_id': 'req-7b2d3f91-7d5d-4cac-93d5-f59b15a809f9'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:41:22.066961 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-7b2d3f91-7d5d-4cac-93d5-f59b15a809f9 admin admin] The token cache is being invalidate because project 7947bfb1d9264cd9a226695ef8f8223c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:41:22.067275 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-7b2d3f91-7d5d-4cac-93d5-f59b15a809f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:41:22.067906 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-7b2d3f91-7d5d-4cac-93d5-f59b15a809f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:41:22.069386 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1256/2503] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:41:21 2022] DELETE /identity/v3/projects/7947bfb1d9264cd9a226695ef8f8223c => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:41:22.074189 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deec1a9-7bc9-460b-9462-c83015c818dd None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:22.074492 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deec1a9-7bc9-460b-9462-c83015c818dd None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:22.074751 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deec1a9-7bc9-460b-9462-c83015c818dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:22.095119 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-7deec1a9-7bc9-460b-9462-c83015c818dd None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:41:22.096754 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-7deec1a9-7bc9-460b-9462-c83015c818dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:41:22.156392 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1248/2504] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:41:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:41:22.166549 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b394eca9-7ee9-4e83-80da-5c065f91bf6c None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:22.166858 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b394eca9-7ee9-4e83-80da-5c065f91bf6c None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:22.167128 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b394eca9-7ee9-4e83-80da-5c065f91bf6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:22.186661 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-b394eca9-7ee9-4e83-80da-5c065f91bf6c None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:41:22.187873 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-b394eca9-7ee9-4e83-80da-5c065f91bf6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:41:22.195551 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1257/2505] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:41:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:41:22.199684 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deec1a9-7bc9-460b-9462-c83015c818dd None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:41:22.210074 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deec1a9-7bc9-460b-9462-c83015c818dd None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:41:22.214633 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c782b81-1f56-4ecc-8c3f-e8c674dc65d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:41:22.215261 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c782b81-1f56-4ecc-8c3f-e8c674dc65d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:22.215541 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c782b81-1f56-4ecc-8c3f-e8c674dc65d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:22.215811 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c782b81-1f56-4ecc-8c3f-e8c674dc65d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:22.216445 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c782b81-1f56-4ecc-8c3f-e8c674dc65d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:41:22.216743 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c782b81-1f56-4ecc-8c3f-e8c674dc65d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:41:22.218307 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c782b81-1f56-4ecc-8c3f-e8c674dc65d9 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:41:22.229771 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1249/2506] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Oct 31 12:41:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:41:22.233871 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b394eca9-7ee9-4e83-80da-5c065f91bf6c None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:41:22.244048 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b394eca9-7ee9-4e83-80da-5c065f91bf6c None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:41:22.249006 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62086b01-e70a-4346-9308-f606eb157ff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:41:22.249615 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62086b01-e70a-4346-9308-f606eb157ff6 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:22.249889 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62086b01-e70a-4346-9308-f606eb157ff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:22.250156 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62086b01-e70a-4346-9308-f606eb157ff6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:22.251425 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62086b01-e70a-4346-9308-f606eb157ff6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:41:22.253356 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62086b01-e70a-4346-9308-f606eb157ff6 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:41:22.279955 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1258/2507] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:41:22 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:41:22.284138 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c782b81-1f56-4ecc-8c3f-e8c674dc65d9 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:41:22.294134 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c782b81-1f56-4ecc-8c3f-e8c674dc65d9 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:41:22.302398 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:41:22.351423 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ad9d79-b276-4757-81a3-82e6a3e17941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:41:22.352046 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ad9d79-b276-4757-81a3-82e6a3e17941 admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:22.352351 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ad9d79-b276-4757-81a3-82e6a3e17941 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:22.352522 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ad9d79-b276-4757-81a3-82e6a3e17941 admin admin] PATH_INFO: `/v3/users` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:22.353144 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ad9d79-b276-4757-81a3-82e6a3e17941 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:41:22.355038 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ad9d79-b276-4757-81a3-82e6a3e17941 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:41:22.378706 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1250/2508] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:41:22 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:41:22.381903 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62086b01-e70a-4346-9308-f606eb157ff6 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:41:22.393067 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62086b01-e70a-4346-9308-f606eb157ff6 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:41:22.400502 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d6776d-31d7-4006-bd38-28bd5f3fbc75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:41:22.401225 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d6776d-31d7-4006-bd38-28bd5f3fbc75 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:22.401513 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d6776d-31d7-4006-bd38-28bd5f3fbc75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:22.401788 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d6776d-31d7-4006-bd38-28bd5f3fbc75 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:22.402437 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d6776d-31d7-4006-bd38-28bd5f3fbc75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:41:22.404055 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d6776d-31d7-4006-bd38-28bd5f3fbc75 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:41:22.415477 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1259/2509] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:41:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:41:22.419472 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ad9d79-b276-4757-81a3-82e6a3e17941 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:41:22.430689 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ad9d79-b276-4757-81a3-82e6a3e17941 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:41:22.435569 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc6a8df-f1fc-44dd-824b-6550b030a7cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:41:22.436248 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc6a8df-f1fc-44dd-824b-6550b030a7cd admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:22.436530 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc6a8df-f1fc-44dd-824b-6550b030a7cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:22.436804 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc6a8df-f1fc-44dd-824b-6550b030a7cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:22.437448 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc6a8df-f1fc-44dd-824b-6550b030a7cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:41:22.441957 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc6a8df-f1fc-44dd-824b-6550b030a7cd admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:41:22.452062 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1251/2510] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:41:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:41:22.456178 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d6776d-31d7-4006-bd38-28bd5f3fbc75 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:41:22.473494 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d6776d-31d7-4006-bd38-28bd5f3fbc75 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:41:22.478866 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a23b9e0-808c-4140-8cbb-8a9f3018b6ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:41:22.479605 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a23b9e0-808c-4140-8cbb-8a9f3018b6ee admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:22.479881 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a23b9e0-808c-4140-8cbb-8a9f3018b6ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:22.480151 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a23b9e0-808c-4140-8cbb-8a9f3018b6ee admin admin] PATH_INFO: `/v3/projects/d49947a82d974a55a8cc7a6ad7b9c56e/users/77385d8c61994d5da4083775d202a93d/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:22.496291 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a23b9e0-808c-4140-8cbb-8a9f3018b6ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=d49947a82d974a55a8cc7a6ad7b9c56e, user_id=77385d8c61994d5da4083775d202a93d, role_id=ca1a5695f8bd499aa0b8cedaeecd2cc6)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:41:22.498897 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a23b9e0-808c-4140-8cbb-8a9f3018b6ee admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:41:22.512248 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1260/2511] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:41:22 2022] PUT /identity/v3/projects/d49947a82d974a55a8cc7a6ad7b9c56e/users/77385d8c61994d5da4083775d202a93d/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 31 12:41:22.517568 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c51740b-4240-4de2-a1cb-0cdd1e8c8fbe None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:22.517843 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c51740b-4240-4de2-a1cb-0cdd1e8c8fbe None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:22.518113 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c51740b-4240-4de2-a1cb-0cdd1e8c8fbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:22.537140 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-9c51740b-4240-4de2-a1cb-0cdd1e8c8fbe None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:41:22.538438 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-9c51740b-4240-4de2-a1cb-0cdd1e8c8fbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:41:22.596745 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1252/2512] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:41:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:41:22.606411 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a23b9e0-808c-4140-8cbb-8a9f3018b6ee admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:41:22.608101 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-6a23b9e0-808c-4140-8cbb-8a9f3018b6ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:41:22.623350 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:41:22.670546 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a23b9e0-808c-4140-8cbb-8a9f3018b6ee admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:41:22.675354 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50b187d-2c45-4ad0-b08d-dfc6f6c75087 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:41:22.675992 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50b187d-2c45-4ad0-b08d-dfc6f6c75087 service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:22.676272 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50b187d-2c45-4ad0-b08d-dfc6f6c75087 service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:22.676544 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50b187d-2c45-4ad0-b08d-dfc6f6c75087 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:22.685319 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50b187d-2c45-4ad0-b08d-dfc6f6c75087 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:41:22.686732 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50b187d-2c45-4ad0-b08d-dfc6f6c75087 service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:41:22.696616 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1261/2513] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:41:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:41:23.287745 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3d1229-9c0f-4b71-8adf-47d0b4108357 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:23.288035 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3d1229-9c0f-4b71-8adf-47d0b4108357 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:23.288316 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3d1229-9c0f-4b71-8adf-47d0b4108357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:23.316521 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-6c3d1229-9c0f-4b71-8adf-47d0b4108357 None None] MFA Rules not processed for user `77385d8c61994d5da4083775d202a93d`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:41:23.317730 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-6c3d1229-9c0f-4b71-8adf-47d0b4108357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:41:23.384076 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1253/2514] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:41:23 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:41:23.396129 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50b187d-2c45-4ad0-b08d-dfc6f6c75087 service neutron] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:41:23.397614 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-f50b187d-2c45-4ad0-b08d-dfc6f6c75087 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:41:23.455730 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50b187d-2c45-4ad0-b08d-dfc6f6c75087 service neutron] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:41:23.459357 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f11842-c9d0-4478-a5b9-5b4f7e2288fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d6d7ea077d49e8a66c6c6762499702', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:41:23.459973 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f11842-c9d0-4478-a5b9-5b4f7e2288fb service cinder] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:23.460251 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f11842-c9d0-4478-a5b9-5b4f7e2288fb service cinder] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:23.460522 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f11842-c9d0-4478-a5b9-5b4f7e2288fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:23.469144 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f11842-c9d0-4478-a5b9-5b4f7e2288fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:41:23.471463 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f11842-c9d0-4478-a5b9-5b4f7e2288fb service cinder] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:41:23.481650 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1262/2515] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:41:23 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:41:24.881081 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c329786-5d1d-48d6-a9e4-99045f24c301 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:24.881543 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c329786-5d1d-48d6-a9e4-99045f24c301 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:24.881654 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c329786-5d1d-48d6-a9e4-99045f24c301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:24.900348 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-1c329786-5d1d-48d6-a9e4-99045f24c301 None None] MFA Rules not processed for user `77385d8c61994d5da4083775d202a93d`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:41:24.901517 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-1c329786-5d1d-48d6-a9e4-99045f24c301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:41:24.909164 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1254/2516] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:41:24 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:41:33.477561 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=46 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:41:34.895436 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=91,keystone:INSERT=3 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:41:39.732059 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f11842-c9d0-4478-a5b9-5b4f7e2288fb service cinder] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:41:39.733447 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-03f11842-c9d0-4478-a5b9-5b4f7e2288fb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:41:39.746929 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f11842-c9d0-4478-a5b9-5b4f7e2288fb service cinder] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:41:39.752152 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702e2ad9-bca5-4f82-ac7f-e8586924de9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:41:39.752761 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702e2ad9-bca5-4f82-ac7f-e8586924de9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:39.753035 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702e2ad9-bca5-4f82-ac7f-e8586924de9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:39.753329 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702e2ad9-bca5-4f82-ac7f-e8586924de9c admin admin] PATH_INFO: `/v3/users/fdf8ee32608a492d91528bec83e86d94` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:39.754426 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702e2ad9-bca5-4f82-ac7f-e8586924de9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=fdf8ee32608a492d91528bec83e86d94)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:41:39.756322 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702e2ad9-bca5-4f82-ac7f-e8586924de9c admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:41:39.802441 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-702e2ad9-bca5-4f82-ac7f-e8586924de9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fdf8ee32608a492d91528bec83e86d94'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:41:39.810283 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-702e2ad9-bca5-4f82-ac7f-e8586924de9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fdf8ee32608a492d91528bec83e86d94'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:41:39.810873 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-702e2ad9-bca5-4f82-ac7f-e8586924de9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fdf8ee32608a492d91528bec83e86d94'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:41:39.811483 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-702e2ad9-bca5-4f82-ac7f-e8586924de9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fdf8ee32608a492d91528bec83e86d94'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:41:39.824041 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-702e2ad9-bca5-4f82-ac7f-e8586924de9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fdf8ee32608a492d91528bec83e86d94'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:41:39.836531 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1263/2517] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:41:39 2022] DELETE /identity/v3/users/fdf8ee32608a492d91528bec83e86d94 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:41:39.840917 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbb10c0-d9ef-4002-9a2a-d299f9835c1f None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:39.841397 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbb10c0-d9ef-4002-9a2a-d299f9835c1f None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:39.841496 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbb10c0-d9ef-4002-9a2a-d299f9835c1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:39.861714 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-bcbb10c0-d9ef-4002-9a2a-d299f9835c1f None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:41:39.863340 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-bcbb10c0-d9ef-4002-9a2a-d299f9835c1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:41:39.922902 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1255/2518] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:41:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 31 12:41:39.934115 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702e2ad9-bca5-4f82-ac7f-e8586924de9c admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:41:39.935646 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-702e2ad9-bca5-4f82-ac7f-e8586924de9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:41:39.994332 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702e2ad9-bca5-4f82-ac7f-e8586924de9c admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:41:39.999596 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494884f0-8064-44ef-910f-91e308dab264 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:41:40.000731 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494884f0-8064-44ef-910f-91e308dab264 service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:40.001011 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494884f0-8064-44ef-910f-91e308dab264 service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:40.001320 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494884f0-8064-44ef-910f-91e308dab264 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:40.012213 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494884f0-8064-44ef-910f-91e308dab264 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:41:40.013624 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494884f0-8064-44ef-910f-91e308dab264 service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:41:40.024426 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1264/2519] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:41:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:41:40.286056 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbb10c0-d9ef-4002-9a2a-d299f9835c1f None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:41:40.287506 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-bcbb10c0-d9ef-4002-9a2a-d299f9835c1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:41:40.300004 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbb10c0-d9ef-4002-9a2a-d299f9835c1f None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:41:40.304582 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3da354-8d59-433b-aad9-5b345248402c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:41:40.305212 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3da354-8d59-433b-aad9-5b345248402c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:40.305489 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3da354-8d59-433b-aad9-5b345248402c admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:40.305758 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3da354-8d59-433b-aad9-5b345248402c admin admin] PATH_INFO: `/v3/projects/871c528fb3a24f17a3ea12bce679cffe` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:40.307354 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3da354-8d59-433b-aad9-5b345248402c admin admin] RBAC: Authorizing `identity:delete_project(project_id=871c528fb3a24f17a3ea12bce679cffe)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:41:40.308793 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3da354-8d59-433b-aad9-5b345248402c admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:41:40.355359 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-8b3da354-8d59-433b-aad9-5b345248402c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '871c528fb3a24f17a3ea12bce679cffe', 'request_id': 'req-8b3da354-8d59-433b-aad9-5b345248402c'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:41:40.361397 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-8b3da354-8d59-433b-aad9-5b345248402c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '871c528fb3a24f17a3ea12bce679cffe', 'request_id': 'req-8b3da354-8d59-433b-aad9-5b345248402c'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:41:40.368404 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-8b3da354-8d59-433b-aad9-5b345248402c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '871c528fb3a24f17a3ea12bce679cffe', 'request_id': 'req-8b3da354-8d59-433b-aad9-5b345248402c'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:41:40.381109 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-8b3da354-8d59-433b-aad9-5b345248402c admin admin] The token cache is being invalidate because project 871c528fb3a24f17a3ea12bce679cffe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:41:40.381383 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-8b3da354-8d59-433b-aad9-5b345248402c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:41:40.381999 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-8b3da354-8d59-433b-aad9-5b345248402c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:41:40.383490 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1256/2520] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:41:40 2022] DELETE /identity/v3/projects/871c528fb3a24f17a3ea12bce679cffe => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:41:40.387490 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494884f0-8064-44ef-910f-91e308dab264 service neutron] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:41:40.397621 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494884f0-8064-44ef-910f-91e308dab264 service neutron] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:41:40.453740 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a369d7f9-d1ae-4676-87b0-492db8ad734f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:41:40.454371 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a369d7f9-d1ae-4676-87b0-492db8ad734f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:40.454649 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a369d7f9-d1ae-4676-87b0-492db8ad734f admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:40.454920 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a369d7f9-d1ae-4676-87b0-492db8ad734f admin admin] PATH_INFO: `/v3/users/d9a2c5c4148941029b52713a30c39122` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:40.456066 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a369d7f9-d1ae-4676-87b0-492db8ad734f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9a2c5c4148941029b52713a30c39122)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:41:40.457956 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a369d7f9-d1ae-4676-87b0-492db8ad734f admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:41:40.473232 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:41:40.503883 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-a369d7f9-d1ae-4676-87b0-492db8ad734f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9a2c5c4148941029b52713a30c39122'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:41:40.511411 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-a369d7f9-d1ae-4676-87b0-492db8ad734f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9a2c5c4148941029b52713a30c39122'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:41:40.512047 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-a369d7f9-d1ae-4676-87b0-492db8ad734f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9a2c5c4148941029b52713a30c39122'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:41:40.512790 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-a369d7f9-d1ae-4676-87b0-492db8ad734f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9a2c5c4148941029b52713a30c39122'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:41:40.524714 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-a369d7f9-d1ae-4676-87b0-492db8ad734f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9a2c5c4148941029b52713a30c39122'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:41:40.537121 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1265/2521] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:41:40 2022] DELETE /identity/v3/users/d9a2c5c4148941029b52713a30c39122 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:41:40.815184 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3da354-8d59-433b-aad9-5b345248402c admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:41:40.816659 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-8b3da354-8d59-433b-aad9-5b345248402c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:41:40.837315 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3da354-8d59-433b-aad9-5b345248402c admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:41:40.904425 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f52041-4c53-41f9-9e84-f507862b5066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:41:40.905057 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f52041-4c53-41f9-9e84-f507862b5066 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:40.905331 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f52041-4c53-41f9-9e84-f507862b5066 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:40.905600 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f52041-4c53-41f9-9e84-f507862b5066 admin admin] PATH_INFO: `/v3/projects/de13e8ffdfde4a07a19471ed7e5a5e1f` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:40.906761 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f52041-4c53-41f9-9e84-f507862b5066 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de13e8ffdfde4a07a19471ed7e5a5e1f)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:41:40.908849 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f52041-4c53-41f9-9e84-f507862b5066 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:41:40.941949 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:41:40.961843 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-c5f52041-4c53-41f9-9e84-f507862b5066 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de13e8ffdfde4a07a19471ed7e5a5e1f', 'request_id': 'req-c5f52041-4c53-41f9-9e84-f507862b5066'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:41:40.968780 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-c5f52041-4c53-41f9-9e84-f507862b5066 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de13e8ffdfde4a07a19471ed7e5a5e1f', 'request_id': 'req-c5f52041-4c53-41f9-9e84-f507862b5066'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:41:40.976906 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-c5f52041-4c53-41f9-9e84-f507862b5066 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de13e8ffdfde4a07a19471ed7e5a5e1f', 'request_id': 'req-c5f52041-4c53-41f9-9e84-f507862b5066'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:41:40.989656 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-c5f52041-4c53-41f9-9e84-f507862b5066 admin admin] The token cache is being invalidate because project de13e8ffdfde4a07a19471ed7e5a5e1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:41:40.989947 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-c5f52041-4c53-41f9-9e84-f507862b5066 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:41:40.990602 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-c5f52041-4c53-41f9-9e84-f507862b5066 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:41:40.992388 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1257/2522] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:41:40 2022] DELETE /identity/v3/projects/de13e8ffdfde4a07a19471ed7e5a5e1f => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:41:40.997082 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3e5590-9ad3-4a73-920d-cb744310a39b None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:40.997538 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3e5590-9ad3-4a73-920d-cb744310a39b None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:40.997622 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3e5590-9ad3-4a73-920d-cb744310a39b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:41.020477 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-fe3e5590-9ad3-4a73-920d-cb744310a39b None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:41:41.021880 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-fe3e5590-9ad3-4a73-920d-cb744310a39b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:41:41.107646 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1266/2523] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:41:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:41:41.182485 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0242e0-0390-4964-99f1-6bed0c397868 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:41.182882 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0242e0-0390-4964-99f1-6bed0c397868 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:41.183243 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0242e0-0390-4964-99f1-6bed0c397868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:41.205310 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-6b0242e0-0390-4964-99f1-6bed0c397868 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:41:41.206627 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-6b0242e0-0390-4964-99f1-6bed0c397868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:41:41.216341 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1258/2524] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:41:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:41:41.231385 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3e5590-9ad3-4a73-920d-cb744310a39b None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:41:41.231502 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3e5590-9ad3-4a73-920d-cb744310a39b None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:41:41.236874 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa155f93-15ca-4768-9cc4-5270c23e579a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:41:41.237567 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa155f93-15ca-4768-9cc4-5270c23e579a admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:41.237846 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa155f93-15ca-4768-9cc4-5270c23e579a admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:41.238113 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa155f93-15ca-4768-9cc4-5270c23e579a admin admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:41.238751 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa155f93-15ca-4768-9cc4-5270c23e579a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:41:41.239047 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa155f93-15ca-4768-9cc4-5270c23e579a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:41:41.240789 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa155f93-15ca-4768-9cc4-5270c23e579a admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:41:41.253153 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1267/2525] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Oct 31 12:41:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 31 12:41:41.257589 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0242e0-0390-4964-99f1-6bed0c397868 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:41:41.267833 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0242e0-0390-4964-99f1-6bed0c397868 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:41:41.272779 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52f9015-6285-4ce8-9321-36dc5ef9ab21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:41:41.273528 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52f9015-6285-4ce8-9321-36dc5ef9ab21 admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:41.273811 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52f9015-6285-4ce8-9321-36dc5ef9ab21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:41.274099 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52f9015-6285-4ce8-9321-36dc5ef9ab21 admin admin] PATH_INFO: `/v3/projects` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:41.274733 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52f9015-6285-4ce8-9321-36dc5ef9ab21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:41:41.276698 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52f9015-6285-4ce8-9321-36dc5ef9ab21 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:41:41.304697 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1259/2526] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:41:41 2022] POST /identity/v3/projects => generated 387 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:41:41.308742 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa155f93-15ca-4768-9cc4-5270c23e579a admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:41:41.319249 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa155f93-15ca-4768-9cc4-5270c23e579a admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:41:41.374253 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8edda4-ce99-4260-b50c-0f40d5c10a9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:41:41.375584 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8edda4-ce99-4260-b50c-0f40d5c10a9d admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:41.375929 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8edda4-ce99-4260-b50c-0f40d5c10a9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:41.376222 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8edda4-ce99-4260-b50c-0f40d5c10a9d admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:41.376849 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8edda4-ce99-4260-b50c-0f40d5c10a9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:41:41.378773 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8edda4-ce99-4260-b50c-0f40d5c10a9d admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:41:41.400367 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1268/2527] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:41:41 2022] POST /identity/v3/users => generated 294 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 31 12:41:41.404488 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52f9015-6285-4ce8-9321-36dc5ef9ab21 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:41:41.415371 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52f9015-6285-4ce8-9321-36dc5ef9ab21 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:41:41.420061 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c755168-1123-4688-bda1-9315d7215d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:41:41.420650 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c755168-1123-4688-bda1-9315d7215d75 admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:41.420929 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c755168-1123-4688-bda1-9315d7215d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:41.421200 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c755168-1123-4688-bda1-9315d7215d75 admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:41.422100 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c755168-1123-4688-bda1-9315d7215d75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:41:41.424105 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c755168-1123-4688-bda1-9315d7215d75 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:41:41.433951 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1260/2528] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:41:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:41:41.438270 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8edda4-ce99-4260-b50c-0f40d5c10a9d admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:41:41.448118 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8edda4-ce99-4260-b50c-0f40d5c10a9d admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:41:41.452744 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53d9ca7-27e2-4b2f-afb6-0e2267b26ea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:41:41.453599 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53d9ca7-27e2-4b2f-afb6-0e2267b26ea5 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:41.453977 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53d9ca7-27e2-4b2f-afb6-0e2267b26ea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:41.454688 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53d9ca7-27e2-4b2f-afb6-0e2267b26ea5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:41.455365 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53d9ca7-27e2-4b2f-afb6-0e2267b26ea5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:41:41.456942 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53d9ca7-27e2-4b2f-afb6-0e2267b26ea5 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:41:41.462230 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:41:41.469381 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1269/2529] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:41:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:41:41.473732 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c755168-1123-4688-bda1-9315d7215d75 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:41:41.483735 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c755168-1123-4688-bda1-9315d7215d75 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:41:41.489003 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2f0f5a-c6ca-41cd-bbf3-3353a449a0c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:41:41.489641 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2f0f5a-c6ca-41cd-bbf3-3353a449a0c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:41.489915 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2f0f5a-c6ca-41cd-bbf3-3353a449a0c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:41.490185 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2f0f5a-c6ca-41cd-bbf3-3353a449a0c7 admin admin] PATH_INFO: `/v3/projects/a260003c9a324bdb884db6ce3a25984a/users/9056ca60980848af81847dc9c994a477/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:41.504347 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2f0f5a-c6ca-41cd-bbf3-3353a449a0c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a260003c9a324bdb884db6ce3a25984a, user_id=9056ca60980848af81847dc9c994a477, role_id=ca1a5695f8bd499aa0b8cedaeecd2cc6)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:41:41.506948 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2f0f5a-c6ca-41cd-bbf3-3353a449a0c7 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:41:41.519843 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1261/2530] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:41:41 2022] PUT /identity/v3/projects/a260003c9a324bdb884db6ce3a25984a/users/9056ca60980848af81847dc9c994a477/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 31 12:41:41.529710 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c8716d-fc3d-43ab-a8b7-6d17cd8a0834 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:41.529970 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c8716d-fc3d-43ab-a8b7-6d17cd8a0834 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:41.530236 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c8716d-fc3d-43ab-a8b7-6d17cd8a0834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:41.558570 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-71c8716d-fc3d-43ab-a8b7-6d17cd8a0834 None None] MFA Rules not processed for user `9056ca60980848af81847dc9c994a477`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:41:41.559784 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-71c8716d-fc3d-43ab-a8b7-6d17cd8a0834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:41:41.626935 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1270/2531] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:41:41 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:41:41.631434 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2f0f5a-c6ca-41cd-bbf3-3353a449a0c7 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:41:41.691690 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2f0f5a-c6ca-41cd-bbf3-3353a449a0c7 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:41:41.696160 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be4c9d3-c19e-446b-9738-9281d10aa228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:41:41.696751 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be4c9d3-c19e-446b-9738-9281d10aa228 admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:41.697032 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be4c9d3-c19e-446b-9738-9281d10aa228 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:41.697297 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be4c9d3-c19e-446b-9738-9281d10aa228 admin admin] PATH_INFO: `/v3/projects` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:41.697903 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be4c9d3-c19e-446b-9738-9281d10aa228 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:41:41.699819 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be4c9d3-c19e-446b-9738-9281d10aa228 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:41:41.726354 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1262/2532] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:41:41 2022] POST /identity/v3/projects => generated 385 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:41:41.730365 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c8716d-fc3d-43ab-a8b7-6d17cd8a0834 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:41:41.741817 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c8716d-fc3d-43ab-a8b7-6d17cd8a0834 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:41:41.800736 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49d7960-72f1-48c3-a11f-19f9d9775d91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:41:41.801348 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49d7960-72f1-48c3-a11f-19f9d9775d91 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:41.801627 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49d7960-72f1-48c3-a11f-19f9d9775d91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:41.801899 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49d7960-72f1-48c3-a11f-19f9d9775d91 admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:41.802518 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49d7960-72f1-48c3-a11f-19f9d9775d91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:41:41.804460 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49d7960-72f1-48c3-a11f-19f9d9775d91 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:41:41.825289 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1271/2533] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:41:41 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:41:41.829804 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be4c9d3-c19e-446b-9738-9281d10aa228 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:41:41.839952 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be4c9d3-c19e-446b-9738-9281d10aa228 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:41:41.845297 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58941633-9970-4a3e-b4f4-e265733a5cf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:41:41.845884 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58941633-9970-4a3e-b4f4-e265733a5cf3 admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:41.846164 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58941633-9970-4a3e-b4f4-e265733a5cf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:41.846434 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58941633-9970-4a3e-b4f4-e265733a5cf3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:41.847063 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58941633-9970-4a3e-b4f4-e265733a5cf3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:41:41.848665 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58941633-9970-4a3e-b4f4-e265733a5cf3 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:41:41.859010 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1263/2534] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:41:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:41:41.862715 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49d7960-72f1-48c3-a11f-19f9d9775d91 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:41:41.872598 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49d7960-72f1-48c3-a11f-19f9d9775d91 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:41:41.877235 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe29cf8f-2284-4d80-a472-a9cfc4a5c65f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:41:41.877824 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe29cf8f-2284-4d80-a472-a9cfc4a5c65f admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:41.878101 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe29cf8f-2284-4d80-a472-a9cfc4a5c65f admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:41.878370 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe29cf8f-2284-4d80-a472-a9cfc4a5c65f admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:41.879008 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe29cf8f-2284-4d80-a472-a9cfc4a5c65f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:41:41.880623 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe29cf8f-2284-4d80-a472-a9cfc4a5c65f admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:41:41.894726 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1272/2535] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:41:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:41:41.896344 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58941633-9970-4a3e-b4f4-e265733a5cf3 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:41:41.907103 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58941633-9970-4a3e-b4f4-e265733a5cf3 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:41:41.912071 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64144584-cc34-4dd2-bc1a-ad56c41fbded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:41:41.912803 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64144584-cc34-4dd2-bc1a-ad56c41fbded admin admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:41.913149 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64144584-cc34-4dd2-bc1a-ad56c41fbded admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:41.913490 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64144584-cc34-4dd2-bc1a-ad56c41fbded admin admin] PATH_INFO: `/v3/projects/5b6547dc77f840c99902c7d22a8e1f3d/users/7828f633bcd1482bba039ee24b7ff4a5/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:41.928446 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64144584-cc34-4dd2-bc1a-ad56c41fbded admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b6547dc77f840c99902c7d22a8e1f3d, user_id=7828f633bcd1482bba039ee24b7ff4a5, role_id=ca1a5695f8bd499aa0b8cedaeecd2cc6)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:41:41.931190 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64144584-cc34-4dd2-bc1a-ad56c41fbded admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:41:41.940900 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:41:41.944982 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1264/2536] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:41:41 2022] PUT /identity/v3/projects/5b6547dc77f840c99902c7d22a8e1f3d/users/7828f633bcd1482bba039ee24b7ff4a5/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:41:41.955783 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feed10c9-36f3-4a36-bca8-8e2b54db9779 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:41.956132 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feed10c9-36f3-4a36-bca8-8e2b54db9779 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:41.956403 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feed10c9-36f3-4a36-bca8-8e2b54db9779 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:41.985014 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-feed10c9-36f3-4a36-bca8-8e2b54db9779 None None] MFA Rules not processed for user `7828f633bcd1482bba039ee24b7ff4a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:41:41.986212 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-feed10c9-36f3-4a36-bca8-8e2b54db9779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:41:42.046714 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:41:42.053543 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1273/2537] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:41:41 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 31 12:41:42.058036 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64144584-cc34-4dd2-bc1a-ad56c41fbded admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:41:42.124183 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64144584-cc34-4dd2-bc1a-ad56c41fbded admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:41:42.129396 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65a294e-28a8-448d-ae31-c1ef2c347e1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:41:42.129997 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65a294e-28a8-448d-ae31-c1ef2c347e1c admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:42.130275 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65a294e-28a8-448d-ae31-c1ef2c347e1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:42.130547 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65a294e-28a8-448d-ae31-c1ef2c347e1c admin admin] PATH_INFO: `/v3/projects` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:42.131183 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65a294e-28a8-448d-ae31-c1ef2c347e1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:41:42.133069 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65a294e-28a8-448d-ae31-c1ef2c347e1c admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:41:42.160400 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1265/2538] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:41:42 2022] POST /identity/v3/projects => generated 387 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:41:42.164471 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feed10c9-36f3-4a36-bca8-8e2b54db9779 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:41:42.175378 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feed10c9-36f3-4a36-bca8-8e2b54db9779 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:41:42.231646 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b6f607-6201-4037-a815-935fbd1be65f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:41:42.232175 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b6f607-6201-4037-a815-935fbd1be65f admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:42.232450 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b6f607-6201-4037-a815-935fbd1be65f admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:42.232719 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b6f607-6201-4037-a815-935fbd1be65f admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:42.233342 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b6f607-6201-4037-a815-935fbd1be65f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:41:42.235465 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b6f607-6201-4037-a815-935fbd1be65f admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:41:42.263255 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1274/2539] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:41:42 2022] POST /identity/v3/users => generated 300 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 31 12:41:42.264394 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65a294e-28a8-448d-ae31-c1ef2c347e1c admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:41:42.276269 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65a294e-28a8-448d-ae31-c1ef2c347e1c admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:41:42.282464 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5426f59-1c01-405d-869b-6ab7daf0a487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:41:42.283195 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5426f59-1c01-405d-869b-6ab7daf0a487 admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:42.283529 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5426f59-1c01-405d-869b-6ab7daf0a487 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:42.283893 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5426f59-1c01-405d-869b-6ab7daf0a487 admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:42.284604 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5426f59-1c01-405d-869b-6ab7daf0a487 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:41:42.286254 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5426f59-1c01-405d-869b-6ab7daf0a487 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:41:42.298576 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1266/2540] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:41:42 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:41:42.302939 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b6f607-6201-4037-a815-935fbd1be65f admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:41:42.313817 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b6f607-6201-4037-a815-935fbd1be65f admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:41:42.318644 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b6aef6-09cb-40cb-80b5-2c70305dbb73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:41:42.319339 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b6aef6-09cb-40cb-80b5-2c70305dbb73 admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:42.319613 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b6aef6-09cb-40cb-80b5-2c70305dbb73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:42.319881 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b6aef6-09cb-40cb-80b5-2c70305dbb73 admin admin] PATH_INFO: `/v3/projects/705a70dd25fb4f68b271393c612648ef/users/c21286b663e74154b6644ba28e6dde4b/roles/176c07b4fe3b4537bdfeb4be21b379c5` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:42.334837 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b6aef6-09cb-40cb-80b5-2c70305dbb73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=705a70dd25fb4f68b271393c612648ef, user_id=c21286b663e74154b6644ba28e6dde4b, role_id=176c07b4fe3b4537bdfeb4be21b379c5)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:41:42.337500 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b6aef6-09cb-40cb-80b5-2c70305dbb73 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:41:42.351559 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1275/2541] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:41:42 2022] PUT /identity/v3/projects/705a70dd25fb4f68b271393c612648ef/users/c21286b663e74154b6644ba28e6dde4b/roles/176c07b4fe3b4537bdfeb4be21b379c5 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:41:42.355274 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5426f59-1c01-405d-869b-6ab7daf0a487 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:41:42.366107 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5426f59-1c01-405d-869b-6ab7daf0a487 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:41:42.423922 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7ac9a0-1f27-4ba7-8c62-d616227ad82d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:41:42.424525 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7ac9a0-1f27-4ba7-8c62-d616227ad82d admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:42.424802 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7ac9a0-1f27-4ba7-8c62-d616227ad82d admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:42.425073 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7ac9a0-1f27-4ba7-8c62-d616227ad82d admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:42.425703 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7ac9a0-1f27-4ba7-8c62-d616227ad82d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:41:42.427336 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7ac9a0-1f27-4ba7-8c62-d616227ad82d admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:41:42.436818 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1267/2542] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:41:42 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:41:42.440972 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b6aef6-09cb-40cb-80b5-2c70305dbb73 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:41:42.451279 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b6aef6-09cb-40cb-80b5-2c70305dbb73 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:41:42.456929 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35372d4-c0ba-4775-b9b9-f80a5d9231fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:41:42.457660 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35372d4-c0ba-4775-b9b9-f80a5d9231fa admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:42.457934 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35372d4-c0ba-4775-b9b9-f80a5d9231fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:42.458199 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35372d4-c0ba-4775-b9b9-f80a5d9231fa admin admin] PATH_INFO: `/v3/projects/705a70dd25fb4f68b271393c612648ef/users/c21286b663e74154b6644ba28e6dde4b/roles/176c07b4fe3b4537bdfeb4be21b379c5` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:42.460113 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35372d4-c0ba-4775-b9b9-f80a5d9231fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=705a70dd25fb4f68b271393c612648ef, user_id=c21286b663e74154b6644ba28e6dde4b, role_id=176c07b4fe3b4537bdfeb4be21b379c5)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:41:42.462708 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35372d4-c0ba-4775-b9b9-f80a5d9231fa admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:41:42.473922 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1276/2543] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:41:42 2022] PUT /identity/v3/projects/705a70dd25fb4f68b271393c612648ef/users/c21286b663e74154b6644ba28e6dde4b/roles/176c07b4fe3b4537bdfeb4be21b379c5 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:41:42.485382 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb6782d-71b8-442b-839d-250384ee91eb None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:42.485598 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb6782d-71b8-442b-839d-250384ee91eb None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:42.485869 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb6782d-71b8-442b-839d-250384ee91eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:42.512604 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-eeb6782d-71b8-442b-839d-250384ee91eb None None] MFA Rules not processed for user `c21286b663e74154b6644ba28e6dde4b`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:41:42.513790 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-eeb6782d-71b8-442b-839d-250384ee91eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:41:42.573521 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:41:42.585079 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1268/2544] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:41:42 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:41:42.596371 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35372d4-c0ba-4775-b9b9-f80a5d9231fa admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:41:42.598088 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-b35372d4-c0ba-4775-b9b9-f80a5d9231fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:41:42.656000 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35372d4-c0ba-4775-b9b9-f80a5d9231fa admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:41:42.660727 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18842be0-680b-48ad-abdf-4abaa428d9d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d6d7ea077d49e8a66c6c6762499702', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:41:42.661324 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18842be0-680b-48ad-abdf-4abaa428d9d5 service cinder] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:42.661603 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18842be0-680b-48ad-abdf-4abaa428d9d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:42.661877 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18842be0-680b-48ad-abdf-4abaa428d9d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:42.670712 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18842be0-680b-48ad-abdf-4abaa428d9d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:41:42.672707 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18842be0-680b-48ad-abdf-4abaa428d9d5 service cinder] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:41:42.682018 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1277/2545] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:41:42 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:41:43.186120 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb6782d-71b8-442b-839d-250384ee91eb None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:41:43.188364 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-eeb6782d-71b8-442b-839d-250384ee91eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:41:43.200196 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb6782d-71b8-442b-839d-250384ee91eb None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:41:43.203692 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0676b582-9e3e-4dc3-b83c-dde208bb8e67 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d6d7ea077d49e8a66c6c6762499702', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:41:43.204289 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0676b582-9e3e-4dc3-b83c-dde208bb8e67 service cinder] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:43.204569 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0676b582-9e3e-4dc3-b83c-dde208bb8e67 service cinder] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:43.204834 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0676b582-9e3e-4dc3-b83c-dde208bb8e67 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:43.264179 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0676b582-9e3e-4dc3-b83c-dde208bb8e67 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:41:43.266075 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0676b582-9e3e-4dc3-b83c-dde208bb8e67 service cinder] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:41:43.275260 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1269/2546] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:41:43 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:41:47.510746 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18842be0-680b-48ad-abdf-4abaa428d9d5 service cinder] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:41:47.519881 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18842be0-680b-48ad-abdf-4abaa428d9d5 service cinder] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:41:47.523676 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b9a8ca-6787-4ff9-92dd-da3a71a18ff9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20d6d7ea077d49e8a66c6c6762499702', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:41:47.524312 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b9a8ca-6787-4ff9-92dd-da3a71a18ff9 service cinder] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:47.524619 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b9a8ca-6787-4ff9-92dd-da3a71a18ff9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:47.524889 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b9a8ca-6787-4ff9-92dd-da3a71a18ff9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:47.577481 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:41:47.587359 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b9a8ca-6787-4ff9-92dd-da3a71a18ff9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:41:47.588821 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b9a8ca-6787-4ff9-92dd-da3a71a18ff9 service cinder] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:41:47.597728 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1278/2547] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:41:47 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:41:49.610342 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0676b582-9e3e-4dc3-b83c-dde208bb8e67 service cinder] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:41:49.611910 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-0676b582-9e3e-4dc3-b83c-dde208bb8e67 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:41:49.677797 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0676b582-9e3e-4dc3-b83c-dde208bb8e67 service cinder] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:41:49.681598 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beed600-1819-487e-b696-afdec41f64b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:41:49.682240 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beed600-1819-487e-b696-afdec41f64b1 service neutron] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:49.682545 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beed600-1819-487e-b696-afdec41f64b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:49.682817 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beed600-1819-487e-b696-afdec41f64b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:49.684657 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-0beed600-1819-487e-b696-afdec41f64b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:41:49.733530 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b9a8ca-6787-4ff9-92dd-da3a71a18ff9 service cinder] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:41:49.755668 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0beed600-1819-487e-b696-afdec41f64b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:41:49.757112 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0beed600-1819-487e-b696-afdec41f64b1 service neutron] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:41:49.768293 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1270/2548] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:41:49 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:41:49.806939 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b9a8ca-6787-4ff9-92dd-da3a71a18ff9 service cinder] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:41:49.811773 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455e178b-e4e1-46bd-b0af-d799c10b7569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:41:49.812409 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455e178b-e4e1-46bd-b0af-d799c10b7569 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:49.812830 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455e178b-e4e1-46bd-b0af-d799c10b7569 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:49.813105 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455e178b-e4e1-46bd-b0af-d799c10b7569 admin admin] PATH_INFO: `/v3/users/9056ca60980848af81847dc9c994a477` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:49.814384 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455e178b-e4e1-46bd-b0af-d799c10b7569 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9056ca60980848af81847dc9c994a477)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:41:49.816834 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455e178b-e4e1-46bd-b0af-d799c10b7569 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:41:49.869122 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-455e178b-e4e1-46bd-b0af-d799c10b7569 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9056ca60980848af81847dc9c994a477'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:41:49.876038 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-455e178b-e4e1-46bd-b0af-d799c10b7569 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9056ca60980848af81847dc9c994a477'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:41:49.876690 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-455e178b-e4e1-46bd-b0af-d799c10b7569 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9056ca60980848af81847dc9c994a477'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:41:49.877344 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-455e178b-e4e1-46bd-b0af-d799c10b7569 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9056ca60980848af81847dc9c994a477'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:41:49.894888 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-455e178b-e4e1-46bd-b0af-d799c10b7569 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9056ca60980848af81847dc9c994a477'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:41:49.908093 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1279/2549] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:41:49 2022] DELETE /identity/v3/users/9056ca60980848af81847dc9c994a477 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 31 12:41:49.912664 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d0461b-0289-4843-b51b-951b501fcaa9 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:49.913049 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d0461b-0289-4843-b51b-951b501fcaa9 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:49.913414 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d0461b-0289-4843-b51b-951b501fcaa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:49.933574 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-16d0461b-0289-4843-b51b-951b501fcaa9 None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:41:49.934783 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-16d0461b-0289-4843-b51b-951b501fcaa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:41:49.980582 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:41:50.001390 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1271/2550] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:41:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:41:50.012567 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455e178b-e4e1-46bd-b0af-d799c10b7569 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:41:50.013955 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-455e178b-e4e1-46bd-b0af-d799c10b7569 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:41:50.079990 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455e178b-e4e1-46bd-b0af-d799c10b7569 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:41:50.083745 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e56325-0457-4bc9-a7ff-18b78b399e9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:41:50.084354 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e56325-0457-4bc9-a7ff-18b78b399e9f service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:50.084630 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e56325-0457-4bc9-a7ff-18b78b399e9f service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:50.084898 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e56325-0457-4bc9-a7ff-18b78b399e9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:50.096719 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e56325-0457-4bc9-a7ff-18b78b399e9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:41:50.098817 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e56325-0457-4bc9-a7ff-18b78b399e9f service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:41:50.109163 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1280/2551] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:41:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:41:50.414543 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d0461b-0289-4843-b51b-951b501fcaa9 None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:41:50.416030 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-16d0461b-0289-4843-b51b-951b501fcaa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:41:50.429050 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d0461b-0289-4843-b51b-951b501fcaa9 None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:41:50.435651 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec611a4-b563-47ad-987f-d889bea1cdba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:41:50.436303 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec611a4-b563-47ad-987f-d889bea1cdba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:50.436592 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec611a4-b563-47ad-987f-d889bea1cdba admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:50.437256 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec611a4-b563-47ad-987f-d889bea1cdba admin admin] PATH_INFO: `/v3/projects/a260003c9a324bdb884db6ce3a25984a` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:50.438757 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec611a4-b563-47ad-987f-d889bea1cdba admin admin] RBAC: Authorizing `identity:delete_project(project_id=a260003c9a324bdb884db6ce3a25984a)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:41:50.442539 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec611a4-b563-47ad-987f-d889bea1cdba admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:41:50.501050 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-6ec611a4-b563-47ad-987f-d889bea1cdba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a260003c9a324bdb884db6ce3a25984a', 'request_id': 'req-6ec611a4-b563-47ad-987f-d889bea1cdba'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:41:50.507496 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-6ec611a4-b563-47ad-987f-d889bea1cdba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a260003c9a324bdb884db6ce3a25984a', 'request_id': 'req-6ec611a4-b563-47ad-987f-d889bea1cdba'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:41:50.517889 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-6ec611a4-b563-47ad-987f-d889bea1cdba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a260003c9a324bdb884db6ce3a25984a', 'request_id': 'req-6ec611a4-b563-47ad-987f-d889bea1cdba'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:41:50.534180 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-6ec611a4-b563-47ad-987f-d889bea1cdba admin admin] The token cache is being invalidate because project a260003c9a324bdb884db6ce3a25984a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:41:50.534475 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-6ec611a4-b563-47ad-987f-d889bea1cdba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:41:50.535317 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-6ec611a4-b563-47ad-987f-d889bea1cdba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:41:50.536937 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1272/2552] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:41:50 2022] DELETE /identity/v3/projects/a260003c9a324bdb884db6ce3a25984a => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:41:50.540930 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e56325-0457-4bc9-a7ff-18b78b399e9f service neutron] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:41:50.552106 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e56325-0457-4bc9-a7ff-18b78b399e9f service neutron] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:41:50.564773 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:41:50.622069 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4bee9f-dad5-442d-a1cb-f26ad330ae80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:41:50.622637 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4bee9f-dad5-442d-a1cb-f26ad330ae80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:50.622939 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4bee9f-dad5-442d-a1cb-f26ad330ae80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:50.623240 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4bee9f-dad5-442d-a1cb-f26ad330ae80 admin admin] PATH_INFO: `/v3/users/7828f633bcd1482bba039ee24b7ff4a5` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:50.624430 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4bee9f-dad5-442d-a1cb-f26ad330ae80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7828f633bcd1482bba039ee24b7ff4a5)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:41:50.626461 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4bee9f-dad5-442d-a1cb-f26ad330ae80 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:41:50.683272 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-de4bee9f-dad5-442d-a1cb-f26ad330ae80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7828f633bcd1482bba039ee24b7ff4a5'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:41:50.690932 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-de4bee9f-dad5-442d-a1cb-f26ad330ae80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7828f633bcd1482bba039ee24b7ff4a5'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:41:50.691676 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-de4bee9f-dad5-442d-a1cb-f26ad330ae80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7828f633bcd1482bba039ee24b7ff4a5'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:41:50.692348 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-de4bee9f-dad5-442d-a1cb-f26ad330ae80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7828f633bcd1482bba039ee24b7ff4a5'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:41:50.706310 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-de4bee9f-dad5-442d-a1cb-f26ad330ae80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7828f633bcd1482bba039ee24b7ff4a5'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:41:50.720921 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1281/2553] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:41:50 2022] DELETE /identity/v3/users/7828f633bcd1482bba039ee24b7ff4a5 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:41:51.025488 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec611a4-b563-47ad-987f-d889bea1cdba admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:41:51.026963 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-6ec611a4-b563-47ad-987f-d889bea1cdba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:41:51.050865 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec611a4-b563-47ad-987f-d889bea1cdba admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:41:51.112737 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e592299-6b6f-4707-9e3c-4c0369c5f206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:41:51.113456 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e592299-6b6f-4707-9e3c-4c0369c5f206 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:51.113692 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e592299-6b6f-4707-9e3c-4c0369c5f206 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:51.113970 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e592299-6b6f-4707-9e3c-4c0369c5f206 admin admin] PATH_INFO: `/v3/projects/5b6547dc77f840c99902c7d22a8e1f3d` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:51.115432 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e592299-6b6f-4707-9e3c-4c0369c5f206 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b6547dc77f840c99902c7d22a8e1f3d)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:41:51.117432 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e592299-6b6f-4707-9e3c-4c0369c5f206 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:41:51.172185 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-1e592299-6b6f-4707-9e3c-4c0369c5f206 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b6547dc77f840c99902c7d22a8e1f3d', 'request_id': 'req-1e592299-6b6f-4707-9e3c-4c0369c5f206'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:41:51.178826 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-1e592299-6b6f-4707-9e3c-4c0369c5f206 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b6547dc77f840c99902c7d22a8e1f3d', 'request_id': 'req-1e592299-6b6f-4707-9e3c-4c0369c5f206'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:41:51.188931 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-1e592299-6b6f-4707-9e3c-4c0369c5f206 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b6547dc77f840c99902c7d22a8e1f3d', 'request_id': 'req-1e592299-6b6f-4707-9e3c-4c0369c5f206'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:41:51.197927 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:41:51.203871 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-1e592299-6b6f-4707-9e3c-4c0369c5f206 admin admin] The token cache is being invalidate because project 5b6547dc77f840c99902c7d22a8e1f3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:41:51.204928 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-1e592299-6b6f-4707-9e3c-4c0369c5f206 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:41:51.205724 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-1e592299-6b6f-4707-9e3c-4c0369c5f206 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:41:51.207757 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1273/2554] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:41:51 2022] DELETE /identity/v3/projects/5b6547dc77f840c99902c7d22a8e1f3d => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:41:51.211478 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4bee9f-dad5-442d-a1cb-f26ad330ae80 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:41:51.213079 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-de4bee9f-dad5-442d-a1cb-f26ad330ae80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:41:51.228126 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4bee9f-dad5-442d-a1cb-f26ad330ae80 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:41:51.291112 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9c1884-1b88-4650-9911-234721121a42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:41:51.291794 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9c1884-1b88-4650-9911-234721121a42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:51.292073 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9c1884-1b88-4650-9911-234721121a42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:51.292343 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9c1884-1b88-4650-9911-234721121a42 admin admin] PATH_INFO: `/v3/users/c21286b663e74154b6644ba28e6dde4b` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:51.293470 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9c1884-1b88-4650-9911-234721121a42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c21286b663e74154b6644ba28e6dde4b)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:41:51.295452 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9c1884-1b88-4650-9911-234721121a42 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:41:51.339588 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:41:51.351594 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-bd9c1884-1b88-4650-9911-234721121a42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c21286b663e74154b6644ba28e6dde4b'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:41:51.361493 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-bd9c1884-1b88-4650-9911-234721121a42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c21286b663e74154b6644ba28e6dde4b'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:41:51.362128 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-bd9c1884-1b88-4650-9911-234721121a42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c21286b663e74154b6644ba28e6dde4b'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:41:51.363442 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-bd9c1884-1b88-4650-9911-234721121a42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c21286b663e74154b6644ba28e6dde4b'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:41:51.375863 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-bd9c1884-1b88-4650-9911-234721121a42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c21286b663e74154b6644ba28e6dde4b'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:41:51.388746 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1282/2555] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:41:51 2022] DELETE /identity/v3/users/c21286b663e74154b6644ba28e6dde4b => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 31 12:41:51.691818 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e592299-6b6f-4707-9e3c-4c0369c5f206 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:41:51.694988 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-1e592299-6b6f-4707-9e3c-4c0369c5f206 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:41:51.712442 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e592299-6b6f-4707-9e3c-4c0369c5f206 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:41:51.791624 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dbb94f-2c28-4963-81f7-b5eb41da884c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:41:51.792314 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dbb94f-2c28-4963-81f7-b5eb41da884c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:51.792711 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dbb94f-2c28-4963-81f7-b5eb41da884c admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:51.793182 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dbb94f-2c28-4963-81f7-b5eb41da884c admin admin] PATH_INFO: `/v3/projects/705a70dd25fb4f68b271393c612648ef` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:51.796070 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dbb94f-2c28-4963-81f7-b5eb41da884c admin admin] RBAC: Authorizing `identity:delete_project(project_id=705a70dd25fb4f68b271393c612648ef)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:41:51.798470 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dbb94f-2c28-4963-81f7-b5eb41da884c admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:41:51.864326 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-37dbb94f-2c28-4963-81f7-b5eb41da884c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '705a70dd25fb4f68b271393c612648ef', 'request_id': 'req-37dbb94f-2c28-4963-81f7-b5eb41da884c'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:41:51.872205 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-37dbb94f-2c28-4963-81f7-b5eb41da884c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '705a70dd25fb4f68b271393c612648ef', 'request_id': 'req-37dbb94f-2c28-4963-81f7-b5eb41da884c'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:41:51.885164 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-37dbb94f-2c28-4963-81f7-b5eb41da884c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '705a70dd25fb4f68b271393c612648ef', 'request_id': 'req-37dbb94f-2c28-4963-81f7-b5eb41da884c'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:41:51.899606 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-37dbb94f-2c28-4963-81f7-b5eb41da884c admin admin] The token cache is being invalidate because project 705a70dd25fb4f68b271393c612648ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:41:51.899889 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-37dbb94f-2c28-4963-81f7-b5eb41da884c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:41:51.900541 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-37dbb94f-2c28-4963-81f7-b5eb41da884c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:41:51.902056 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1274/2556] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:41:51 2022] DELETE /identity/v3/projects/705a70dd25fb4f68b271393c612648ef => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:41:51.908217 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc5588e-c8e4-4917-a978-c32573a59b7c None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:51.908612 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc5588e-c8e4-4917-a978-c32573a59b7c None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:51.909046 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc5588e-c8e4-4917-a978-c32573a59b7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:51.932564 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-4dc5588e-c8e4-4917-a978-c32573a59b7c None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:41:51.934016 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-4dc5588e-c8e4-4917-a978-c32573a59b7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:41:52.009363 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1283/2557] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:41:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:41:52.025881 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01126d9-283e-4fc5-b8e3-abf02c39888a None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:52.027037 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01126d9-283e-4fc5-b8e3-abf02c39888a None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:52.027522 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01126d9-283e-4fc5-b8e3-abf02c39888a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:52.052540 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-e01126d9-283e-4fc5-b8e3-abf02c39888a None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:41:52.053924 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-e01126d9-283e-4fc5-b8e3-abf02c39888a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:41:52.063492 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1275/2558] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:41:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:41:52.068500 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc5588e-c8e4-4917-a978-c32573a59b7c None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:41:52.079011 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc5588e-c8e4-4917-a978-c32573a59b7c None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:41:52.084470 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb9d2b5-1eaa-42ff-b227-a4a832558cd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:41:52.085108 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb9d2b5-1eaa-42ff-b227-a4a832558cd6 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:52.085381 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb9d2b5-1eaa-42ff-b227-a4a832558cd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:52.085648 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb9d2b5-1eaa-42ff-b227-a4a832558cd6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:52.086347 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb9d2b5-1eaa-42ff-b227-a4a832558cd6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 31 12:41:52.086643 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb9d2b5-1eaa-42ff-b227-a4a832558cd6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:41:52.090318 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb9d2b5-1eaa-42ff-b227-a4a832558cd6 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:41:52.103887 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1284/2559] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Oct 31 12:41:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 31 12:41:52.108497 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01126d9-283e-4fc5-b8e3-abf02c39888a None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:41:52.118411 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01126d9-283e-4fc5-b8e3-abf02c39888a None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:41:52.127853 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260671c0-507a-413e-872b-91e6dc37e43b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:41:52.129139 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260671c0-507a-413e-872b-91e6dc37e43b admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:52.129541 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260671c0-507a-413e-872b-91e6dc37e43b admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:52.129879 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260671c0-507a-413e-872b-91e6dc37e43b admin admin] PATH_INFO: `/v3/projects` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:52.130445 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260671c0-507a-413e-872b-91e6dc37e43b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:41:52.132371 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260671c0-507a-413e-872b-91e6dc37e43b admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:41:52.170716 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1276/2560] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:41:52 2022] POST /identity/v3/projects => generated 383 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 31 12:41:52.172090 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb9d2b5-1eaa-42ff-b227-a4a832558cd6 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:41:52.183814 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb9d2b5-1eaa-42ff-b227-a4a832558cd6 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:41:52.265037 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672043ae-7d27-469d-93f8-b41229a1b928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:41:52.266033 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672043ae-7d27-469d-93f8-b41229a1b928 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:52.266320 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672043ae-7d27-469d-93f8-b41229a1b928 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:52.266933 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672043ae-7d27-469d-93f8-b41229a1b928 admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:52.267729 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672043ae-7d27-469d-93f8-b41229a1b928 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:41:52.271068 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672043ae-7d27-469d-93f8-b41229a1b928 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:41:52.297542 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1285/2561] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:41:52 2022] POST /identity/v3/users => generated 292 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 31 12:41:52.301845 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260671c0-507a-413e-872b-91e6dc37e43b admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:41:52.313497 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260671c0-507a-413e-872b-91e6dc37e43b admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:41:52.318672 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1287175d-c5ca-47c2-a5fa-0030e94851fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:41:52.319432 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1287175d-c5ca-47c2-a5fa-0030e94851fa admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:52.319710 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1287175d-c5ca-47c2-a5fa-0030e94851fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:52.319979 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1287175d-c5ca-47c2-a5fa-0030e94851fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:52.320640 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1287175d-c5ca-47c2-a5fa-0030e94851fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:41:52.325739 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1287175d-c5ca-47c2-a5fa-0030e94851fa admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:41:52.337136 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1277/2562] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:41:52 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:41:52.338817 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672043ae-7d27-469d-93f8-b41229a1b928 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:41:52.350892 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672043ae-7d27-469d-93f8-b41229a1b928 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:41:52.355819 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb55fc79-862b-4bce-bd18-1d2b33649899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:41:52.356559 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb55fc79-862b-4bce-bd18-1d2b33649899 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:52.356937 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb55fc79-862b-4bce-bd18-1d2b33649899 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:52.357301 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb55fc79-862b-4bce-bd18-1d2b33649899 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:52.358034 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb55fc79-862b-4bce-bd18-1d2b33649899 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:41:52.361631 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb55fc79-862b-4bce-bd18-1d2b33649899 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:41:52.377769 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1286/2563] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:41:52 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 31 12:41:52.379544 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1287175d-c5ca-47c2-a5fa-0030e94851fa admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:41:52.391205 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1287175d-c5ca-47c2-a5fa-0030e94851fa admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:41:52.400499 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16367a8c-4dc5-4e60-8d7b-ef6158ac59e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:41:52.401179 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16367a8c-4dc5-4e60-8d7b-ef6158ac59e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:52.401459 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16367a8c-4dc5-4e60-8d7b-ef6158ac59e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:52.401731 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16367a8c-4dc5-4e60-8d7b-ef6158ac59e9 admin admin] PATH_INFO: `/v3/projects/0637756aceaf4644bf263b5b5d72d9ac/users/5ddf1d02fa2644d280d9dee5b3101366/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:52.417790 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16367a8c-4dc5-4e60-8d7b-ef6158ac59e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0637756aceaf4644bf263b5b5d72d9ac, user_id=5ddf1d02fa2644d280d9dee5b3101366, role_id=ca1a5695f8bd499aa0b8cedaeecd2cc6)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:41:52.420486 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16367a8c-4dc5-4e60-8d7b-ef6158ac59e9 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:41:52.438315 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1278/2564] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:41:52 2022] PUT /identity/v3/projects/0637756aceaf4644bf263b5b5d72d9ac/users/5ddf1d02fa2644d280d9dee5b3101366/roles/ca1a5695f8bd499aa0b8cedaeecd2cc6 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:41:52.443874 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20228de-46f7-4648-86f7-de7a0fea983e None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:52.444249 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20228de-46f7-4648-86f7-de7a0fea983e None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:52.444512 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20228de-46f7-4648-86f7-de7a0fea983e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:52.468680 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-b20228de-46f7-4648-86f7-de7a0fea983e None None] MFA Rules not processed for user `d2ddf15ba0c7488b8e49f974bba93de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:41:52.470053 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-b20228de-46f7-4648-86f7-de7a0fea983e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:41:52.491692 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:41:52.555223 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1287/2565] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:41:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 31 12:41:52.565028 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16367a8c-4dc5-4e60-8d7b-ef6158ac59e9 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:41:52.568158 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-16367a8c-4dc5-4e60-8d7b-ef6158ac59e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:41:52.599856 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:41:52.641597 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16367a8c-4dc5-4e60-8d7b-ef6158ac59e9 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:41:52.645890 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686d0621-2276-4488-a39e-671e4616dc7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:41:52.646499 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686d0621-2276-4488-a39e-671e4616dc7c service neutron] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:52.646783 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686d0621-2276-4488-a39e-671e4616dc7c service neutron] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:52.647048 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686d0621-2276-4488-a39e-671e4616dc7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:52.657337 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686d0621-2276-4488-a39e-671e4616dc7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:41:52.658920 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686d0621-2276-4488-a39e-671e4616dc7c service neutron] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:41:52.669316 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1279/2566] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:41:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:41:56.723541 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca966eac-4ebb-4395-b7ec-6b16c183def5 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:56.728650 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca966eac-4ebb-4395-b7ec-6b16c183def5 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:56.732057 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca966eac-4ebb-4395-b7ec-6b16c183def5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:56.775210 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-ca966eac-4ebb-4395-b7ec-6b16c183def5 None None] MFA Rules not processed for user `5ddf1d02fa2644d280d9dee5b3101366`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:41:56.776559 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-ca966eac-4ebb-4395-b7ec-6b16c183def5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:41:56.848428 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1288/2567] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:41:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:41:56.853182 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686d0621-2276-4488-a39e-671e4616dc7c service neutron] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:41:56.863583 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686d0621-2276-4488-a39e-671e4616dc7c service neutron] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:41:56.868442 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8954c8e5-932d-4541-8d33-c8050f061d04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:41:56.869049 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8954c8e5-932d-4541-8d33-c8050f061d04 admin admin] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:56.869382 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8954c8e5-932d-4541-8d33-c8050f061d04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:56.869610 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8954c8e5-932d-4541-8d33-c8050f061d04 admin admin] PATH_INFO: `/v3/projects` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:56.870236 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8954c8e5-932d-4541-8d33-c8050f061d04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:41:56.872209 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8954c8e5-932d-4541-8d33-c8050f061d04 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:41:56.905700 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1280/2568] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Oct 31 12:41:56 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 31 12:41:56.907232 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca966eac-4ebb-4395-b7ec-6b16c183def5 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:41:56.919185 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca966eac-4ebb-4395-b7ec-6b16c183def5 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:41:56.979517 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407d9ed1-f72e-4e7f-ae90-3b1dd184e82a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:41:56.980133 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d9ed1-f72e-4e7f-ae90-3b1dd184e82a admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:56.980412 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d9ed1-f72e-4e7f-ae90-3b1dd184e82a admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:56.980682 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d9ed1-f72e-4e7f-ae90-3b1dd184e82a admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:56.981335 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407d9ed1-f72e-4e7f-ae90-3b1dd184e82a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:41:56.983294 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407d9ed1-f72e-4e7f-ae90-3b1dd184e82a admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:41:57.008244 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1289/2569] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Oct 31 12:41:56 2022] POST /identity/v3/users => generated 298 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 31 12:41:57.012697 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8954c8e5-932d-4541-8d33-c8050f061d04 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:41:57.024291 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8954c8e5-932d-4541-8d33-c8050f061d04 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:41:57.029748 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ca222b-8960-416a-b809-6adb3d00b1b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:41:57.030460 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ca222b-8960-416a-b809-6adb3d00b1b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:57.030817 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ca222b-8960-416a-b809-6adb3d00b1b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:57.031116 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ca222b-8960-416a-b809-6adb3d00b1b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:57.031923 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ca222b-8960-416a-b809-6adb3d00b1b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:41:57.033595 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ca222b-8960-416a-b809-6adb3d00b1b4 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:41:57.045745 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1281/2570] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:41:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:41:57.049818 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407d9ed1-f72e-4e7f-ae90-3b1dd184e82a admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:41:57.060791 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407d9ed1-f72e-4e7f-ae90-3b1dd184e82a admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:41:57.065686 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93945ad0-7547-4164-9233-cd7b8ed2b25a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:41:57.066369 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93945ad0-7547-4164-9233-cd7b8ed2b25a admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:57.066739 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93945ad0-7547-4164-9233-cd7b8ed2b25a admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:57.067017 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93945ad0-7547-4164-9233-cd7b8ed2b25a admin admin] PATH_INFO: `/v3/projects/0be1ace5cc7842cd98dfc975e23f13fb/users/789ac79c43df4cbab581ceada876d4e5/roles/176c07b4fe3b4537bdfeb4be21b379c5` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:57.083905 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93945ad0-7547-4164-9233-cd7b8ed2b25a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0be1ace5cc7842cd98dfc975e23f13fb, user_id=789ac79c43df4cbab581ceada876d4e5, role_id=176c07b4fe3b4537bdfeb4be21b379c5)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:41:57.086794 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93945ad0-7547-4164-9233-cd7b8ed2b25a admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:41:57.100806 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1290/2571] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:41:57 2022] PUT /identity/v3/projects/0be1ace5cc7842cd98dfc975e23f13fb/users/789ac79c43df4cbab581ceada876d4e5/roles/176c07b4fe3b4537bdfeb4be21b379c5 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 31 12:41:57.104874 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ca222b-8960-416a-b809-6adb3d00b1b4 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:41:57.115807 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ca222b-8960-416a-b809-6adb3d00b1b4 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:41:57.173030 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd923d3a-382d-4441-8f72-e50636e5a7ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:41:57.173638 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd923d3a-382d-4441-8f72-e50636e5a7ca admin admin] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:57.173913 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd923d3a-382d-4441-8f72-e50636e5a7ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:57.174177 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd923d3a-382d-4441-8f72-e50636e5a7ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:57.174811 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd923d3a-382d-4441-8f72-e50636e5a7ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:41:57.177311 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd923d3a-382d-4441-8f72-e50636e5a7ca admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:41:57.187751 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1282/2572] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Oct 31 12:41:57 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 31 12:41:57.192205 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93945ad0-7547-4164-9233-cd7b8ed2b25a admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:41:57.202998 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93945ad0-7547-4164-9233-cd7b8ed2b25a admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:41:57.208777 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c266e24-652a-40d6-94ca-948e4194abb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:41:57.209598 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c266e24-652a-40d6-94ca-948e4194abb3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:41:57.209938 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c266e24-652a-40d6-94ca-948e4194abb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:41:57.210229 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c266e24-652a-40d6-94ca-948e4194abb3 admin admin] PATH_INFO: `/v3/projects/0be1ace5cc7842cd98dfc975e23f13fb/users/789ac79c43df4cbab581ceada876d4e5/roles/176c07b4fe3b4537bdfeb4be21b379c5` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:41:57.212345 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c266e24-652a-40d6-94ca-948e4194abb3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0be1ace5cc7842cd98dfc975e23f13fb, user_id=789ac79c43df4cbab581ceada876d4e5, role_id=176c07b4fe3b4537bdfeb4be21b379c5)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:41:57.214964 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c266e24-652a-40d6-94ca-948e4194abb3 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:41:57.221234 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:41:57.227151 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1291/2573] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Oct 31 12:41:57 2022] PUT /identity/v3/projects/0be1ace5cc7842cd98dfc975e23f13fb/users/789ac79c43df4cbab581ceada876d4e5/roles/176c07b4fe3b4537bdfeb4be21b379c5 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:42:02.146770 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe43fcd-cbf3-4c6c-a93f-8dd172abc984 None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:42:02.147324 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe43fcd-cbf3-4c6c-a93f-8dd172abc984 None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:42:02.147473 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe43fcd-cbf3-4c6c-a93f-8dd172abc984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:42:02.177741 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-1fe43fcd-cbf3-4c6c-a93f-8dd172abc984 None None] MFA Rules not processed for user `789ac79c43df4cbab581ceada876d4e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:42:02.179099 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-1fe43fcd-cbf3-4c6c-a93f-8dd172abc984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:42:02.247801 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1283/2574] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:42:02 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:42:02.261294 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c266e24-652a-40d6-94ca-948e4194abb3 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:42:02.263060 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-3c266e24-652a-40d6-94ca-948e4194abb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:42:02.344920 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c266e24-652a-40d6-94ca-948e4194abb3 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:42:02.348854 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805a91c1-3f35-4fdb-8fe6-430a20e3ec9f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4feaa01c5ed64b65b10d4dece823702c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:42:02.349477 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805a91c1-3f35-4fdb-8fe6-430a20e3ec9f service glance] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:42:02.349791 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805a91c1-3f35-4fdb-8fe6-430a20e3ec9f service glance] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:42:02.350016 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805a91c1-3f35-4fdb-8fe6-430a20e3ec9f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:42:02.405434 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805a91c1-3f35-4fdb-8fe6-430a20e3ec9f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:42:02.406862 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805a91c1-3f35-4fdb-8fe6-430a20e3ec9f service glance] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:42:02.415893 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1292/2575] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:42:02 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:42:02.546200 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ad2b3d-e66a-4d29-a2ef-28138a5c6abf None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:42:02.546489 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ad2b3d-e66a-4d29-a2ef-28138a5c6abf None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:42:02.546766 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ad2b3d-e66a-4d29-a2ef-28138a5c6abf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:42:02.572902 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:42:02.579511 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-f5ad2b3d-e66a-4d29-a2ef-28138a5c6abf None None] MFA Rules not processed for user `bf4faacf7df248d28f32d7014dadcc07`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:42:02.581801 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-f5ad2b3d-e66a-4d29-a2ef-28138a5c6abf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:42:02.655501 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1284/2576] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Oct 31 12:42:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:42:02.701113 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805a91c1-3f35-4fdb-8fe6-430a20e3ec9f service glance] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:42:02.702525 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-805a91c1-3f35-4fdb-8fe6-430a20e3ec9f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:42:02.772091 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805a91c1-3f35-4fdb-8fe6-430a20e3ec9f service glance] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:42:02.777009 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad1b132-a776-4c5a-915b-ae095ad6fcb0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2254a5786c249b4aa79244f7198ae65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:42:02.777610 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad1b132-a776-4c5a-915b-ae095ad6fcb0 service swift] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:42:02.777884 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad1b132-a776-4c5a-915b-ae095ad6fcb0 service swift] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:42:02.778152 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad1b132-a776-4c5a-915b-ae095ad6fcb0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:42:02.786154 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad1b132-a776-4c5a-915b-ae095ad6fcb0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:42:02.787761 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad1b132-a776-4c5a-915b-ae095ad6fcb0 service swift] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:42:02.795543 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1293/2577] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Oct 31 12:42:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 31 12:42:11.886495 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95373d74-bd2a-48ef-8fa5-ddf1030a5c4c None None] REQUEST_METHOD: `POST` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:42:11.887021 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95373d74-bd2a-48ef-8fa5-ddf1030a5c4c None None] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:42:11.887021 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95373d74-bd2a-48ef-8fa5-ddf1030a5c4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:42:11.907661 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.auth.core [None req-95373d74-bd2a-48ef-8fa5-ddf1030a5c4c None None] MFA Rules not processed for user `5ddf1d02fa2644d280d9dee5b3101366`. Rule list: `[]` (Enabled: `True`). {{(pid=81492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:42:11.909373 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-95373d74-bd2a-48ef-8fa5-ddf1030a5c4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:42:11.916421 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1285/2578] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:42:11 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:42:12.792354 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=80 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:42:15.561871 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d383ef21-ba50-4bb3-a79e-acc1f1873dd9 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:42:15.562448 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d383ef21-ba50-4bb3-a79e-acc1f1873dd9 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:42:15.562677 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d383ef21-ba50-4bb3-a79e-acc1f1873dd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:42:15.583083 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-d383ef21-ba50-4bb3-a79e-acc1f1873dd9 None None] MFA Rules not processed for user `bf4faacf7df248d28f32d7014dadcc07`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:42:15.584484 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-d383ef21-ba50-4bb3-a79e-acc1f1873dd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:42:15.592440 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1294/2579] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Oct 31 12:42:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:42:15.604355 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95373d74-bd2a-48ef-8fa5-ddf1030a5c4c None None] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:42:15.616375 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95373d74-bd2a-48ef-8fa5-ddf1030a5c4c None None] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:42:15.622180 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a59d3d-8cdd-4b18-9cb3-e3d016236ea7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2254a5786c249b4aa79244f7198ae65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:42:15.622781 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a59d3d-8cdd-4b18-9cb3-e3d016236ea7 service swift] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:42:15.623056 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a59d3d-8cdd-4b18-9cb3-e3d016236ea7 service swift] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:42:15.623390 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a59d3d-8cdd-4b18-9cb3-e3d016236ea7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:42:15.633045 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a59d3d-8cdd-4b18-9cb3-e3d016236ea7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:42:15.635373 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a59d3d-8cdd-4b18-9cb3-e3d016236ea7 service swift] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:42:15.645778 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1286/2580] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Oct 31 12:42:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 31 12:42:25.577042 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=3 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:42:25.640741 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=34 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:42:40.540590 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd435e8-11a7-422e-bd58-e01f63792253 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:42:40.541099 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd435e8-11a7-422e-bd58-e01f63792253 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:42:40.541195 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd435e8-11a7-422e-bd58-e01f63792253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:42:40.560192 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-1bd435e8-11a7-422e-bd58-e01f63792253 None None] MFA Rules not processed for user `77385d8c61994d5da4083775d202a93d`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:42:40.561384 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-1bd435e8-11a7-422e-bd58-e01f63792253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:42:40.618567 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1295/2581] 10.222.0.15 () {58 vars in 958 bytes} [Mon Oct 31 12:42:40 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:42:45.763670 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a59d3d-8cdd-4b18-9cb3-e3d016236ea7 service swift] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:42:45.765125 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-66a59d3d-8cdd-4b18-9cb3-e3d016236ea7 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:42:45.820803 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a59d3d-8cdd-4b18-9cb3-e3d016236ea7 service swift] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:42:45.824343 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae6a1db-5628-4d84-9404-6d0f366ba262 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:42:45.824935 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae6a1db-5628-4d84-9404-6d0f366ba262 service neutron] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:42:45.825207 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae6a1db-5628-4d84-9404-6d0f366ba262 service neutron] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:42:45.825472 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae6a1db-5628-4d84-9404-6d0f366ba262 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:42:45.827139 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-8ae6a1db-5628-4d84-9404-6d0f366ba262 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:42:45.906650 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae6a1db-5628-4d84-9404-6d0f366ba262 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:42:45.908017 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae6a1db-5628-4d84-9404-6d0f366ba262 service neutron] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:42:45.929123 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1287/2582] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:42:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:42:47.105173 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd435e8-11a7-422e-bd58-e01f63792253 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:42:47.114816 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd435e8-11a7-422e-bd58-e01f63792253 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:42:47.118563 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aff281-e0d0-4e33-948b-2648303c8f70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:42:47.119288 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aff281-e0d0-4e33-948b-2648303c8f70 service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:42:47.119629 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aff281-e0d0-4e33-948b-2648303c8f70 service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:42:47.119965 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aff281-e0d0-4e33-948b-2648303c8f70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:42:47.121722 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-41aff281-e0d0-4e33-948b-2648303c8f70 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:42:47.136387 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41aff281-e0d0-4e33-948b-2648303c8f70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:42:47.137788 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41aff281-e0d0-4e33-948b-2648303c8f70 service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:42:47.149957 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1296/2583] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:42:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:42:48.697319 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae6a1db-5628-4d84-9404-6d0f366ba262 service neutron] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:42:48.706071 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae6a1db-5628-4d84-9404-6d0f366ba262 service neutron] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:42:48.710389 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f4d14f-e57b-48e2-9ad5-ed581b7fd271 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:42:48.710991 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f4d14f-e57b-48e2-9ad5-ed581b7fd271 service neutron] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:42:48.711307 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f4d14f-e57b-48e2-9ad5-ed581b7fd271 service neutron] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:42:48.711583 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f4d14f-e57b-48e2-9ad5-ed581b7fd271 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:42:48.720411 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f4d14f-e57b-48e2-9ad5-ed581b7fd271 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:42:48.721820 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f4d14f-e57b-48e2-9ad5-ed581b7fd271 service neutron] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:42:48.731219 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1288/2584] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:42:48 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:42:57.144171 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=31 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:42:58.727066 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=69 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:43:40.749681 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aff281-e0d0-4e33-948b-2648303c8f70 service neutron] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:43:40.751258 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-41aff281-e0d0-4e33-948b-2648303c8f70 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:43:40.819765 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aff281-e0d0-4e33-948b-2648303c8f70 service neutron] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:43:40.824187 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5c0a77-3af2-469b-8a85-d6027d1705e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:43:40.825119 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5c0a77-3af2-469b-8a85-d6027d1705e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:43:40.825461 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5c0a77-3af2-469b-8a85-d6027d1705e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:43:40.825735 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5c0a77-3af2-469b-8a85-d6027d1705e1 admin admin] PATH_INFO: `/v3/users/77385d8c61994d5da4083775d202a93d` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:43:40.826845 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5c0a77-3af2-469b-8a85-d6027d1705e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=77385d8c61994d5da4083775d202a93d)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:43:40.828764 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5c0a77-3af2-469b-8a85-d6027d1705e1 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:43:40.875743 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-4a5c0a77-3af2-469b-8a85-d6027d1705e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77385d8c61994d5da4083775d202a93d'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:43:40.883814 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-4a5c0a77-3af2-469b-8a85-d6027d1705e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77385d8c61994d5da4083775d202a93d'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:43:40.884552 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-4a5c0a77-3af2-469b-8a85-d6027d1705e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77385d8c61994d5da4083775d202a93d'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:43:40.885314 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-4a5c0a77-3af2-469b-8a85-d6027d1705e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77385d8c61994d5da4083775d202a93d'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:43:40.898016 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-4a5c0a77-3af2-469b-8a85-d6027d1705e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77385d8c61994d5da4083775d202a93d'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:43:40.911562 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1297/2585] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:43:40 2022] DELETE /identity/v3/users/77385d8c61994d5da4083775d202a93d => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:43:41.097140 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f4d14f-e57b-48e2-9ad5-ed581b7fd271 service neutron] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:43:41.110855 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f4d14f-e57b-48e2-9ad5-ed581b7fd271 service neutron] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:43:41.171186 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f89b858-d46f-403b-a75b-eafc1287d4f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:43:41.171919 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f89b858-d46f-403b-a75b-eafc1287d4f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:43:41.172203 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f89b858-d46f-403b-a75b-eafc1287d4f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:43:41.172475 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f89b858-d46f-403b-a75b-eafc1287d4f2 admin admin] PATH_INFO: `/v3/projects/d49947a82d974a55a8cc7a6ad7b9c56e` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:43:41.173616 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f89b858-d46f-403b-a75b-eafc1287d4f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d49947a82d974a55a8cc7a6ad7b9c56e)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:43:41.175543 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f89b858-d46f-403b-a75b-eafc1287d4f2 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:43:41.223502 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-8f89b858-d46f-403b-a75b-eafc1287d4f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd49947a82d974a55a8cc7a6ad7b9c56e', 'request_id': 'req-8f89b858-d46f-403b-a75b-eafc1287d4f2'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:43:41.229553 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-8f89b858-d46f-403b-a75b-eafc1287d4f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd49947a82d974a55a8cc7a6ad7b9c56e', 'request_id': 'req-8f89b858-d46f-403b-a75b-eafc1287d4f2'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:43:41.239907 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-8f89b858-d46f-403b-a75b-eafc1287d4f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd49947a82d974a55a8cc7a6ad7b9c56e', 'request_id': 'req-8f89b858-d46f-403b-a75b-eafc1287d4f2'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:43:41.253157 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-8f89b858-d46f-403b-a75b-eafc1287d4f2 admin admin] The token cache is being invalidate because project d49947a82d974a55a8cc7a6ad7b9c56e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:43:41.253455 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-8f89b858-d46f-403b-a75b-eafc1287d4f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:43:41.254105 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-8f89b858-d46f-403b-a75b-eafc1287d4f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:43:41.261073 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1289/2586] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:43:41 2022] DELETE /identity/v3/projects/d49947a82d974a55a8cc7a6ad7b9c56e => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:43:41.786633 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5c0a77-3af2-469b-8a85-d6027d1705e1 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:43:41.788570 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-4a5c0a77-3af2-469b-8a85-d6027d1705e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:43:41.872755 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5c0a77-3af2-469b-8a85-d6027d1705e1 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:43:41.888547 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:43:41.960569 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90664f4b-2b72-4454-abc0-17099991d0b3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99889b1dab0b471d9c77887cffefc652', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:43:41.961241 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90664f4b-2b72-4454-abc0-17099991d0b3 service nova] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:43:41.961521 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90664f4b-2b72-4454-abc0-17099991d0b3 service nova] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:43:41.961791 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90664f4b-2b72-4454-abc0-17099991d0b3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:43:41.963676 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-90664f4b-2b72-4454-abc0-17099991d0b3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:43:41.976856 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90664f4b-2b72-4454-abc0-17099991d0b3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:43:41.978540 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90664f4b-2b72-4454-abc0-17099991d0b3 service nova] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:43:41.990594 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1298/2587] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:43:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 31 12:43:43.084589 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f89b858-d46f-403b-a75b-eafc1287d4f2 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:43:43.085928 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-8f89b858-d46f-403b-a75b-eafc1287d4f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:43:43.144625 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f89b858-d46f-403b-a75b-eafc1287d4f2 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:43:43.148285 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb45a48-fd92-4ae0-8784-9e8b58468e5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:43:43.148884 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb45a48-fd92-4ae0-8784-9e8b58468e5f service neutron] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:43:43.149160 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb45a48-fd92-4ae0-8784-9e8b58468e5f service neutron] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:43:43.149429 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb45a48-fd92-4ae0-8784-9e8b58468e5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:43:43.151126 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-0fb45a48-fd92-4ae0-8784-9e8b58468e5f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:43:43.161175 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb45a48-fd92-4ae0-8784-9e8b58468e5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:43:43.162617 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb45a48-fd92-4ae0-8784-9e8b58468e5f service neutron] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:43:43.171479 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1290/2588] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:43:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:43:51.985640 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=26 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:43:53.167016 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=77,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:43:54.040135 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90664f4b-2b72-4454-abc0-17099991d0b3 service nova] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:43:54.041538 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-90664f4b-2b72-4454-abc0-17099991d0b3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:43:54.053885 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90664f4b-2b72-4454-abc0-17099991d0b3 service nova] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:43:54.057507 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367c1169-a2fa-4e62-b43c-998cc8f83fb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25e9c7d1e4714bf18a1ced84855423d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:43:54.058103 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367c1169-a2fa-4e62-b43c-998cc8f83fb1 service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:43:54.058383 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367c1169-a2fa-4e62-b43c-998cc8f83fb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:43:54.058656 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367c1169-a2fa-4e62-b43c-998cc8f83fb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:43:54.060343 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-367c1169-a2fa-4e62-b43c-998cc8f83fb1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:43:54.122888 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367c1169-a2fa-4e62-b43c-998cc8f83fb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:43:54.124880 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367c1169-a2fa-4e62-b43c-998cc8f83fb1 service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:43:54.134470 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1299/2589] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:43:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:44:00.651907 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb45a48-fd92-4ae0-8784-9e8b58468e5f service neutron] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:44:00.653363 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-0fb45a48-fd92-4ae0-8784-9e8b58468e5f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:44:00.676810 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb45a48-fd92-4ae0-8784-9e8b58468e5f service neutron] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:44:00.681356 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031d8bea-cd16-492e-91f1-cf0d0abc5b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:44:00.682021 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031d8bea-cd16-492e-91f1-cf0d0abc5b2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:44:00.682300 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031d8bea-cd16-492e-91f1-cf0d0abc5b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:44:00.682568 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031d8bea-cd16-492e-91f1-cf0d0abc5b2e admin admin] PATH_INFO: `/v3/users/49b76d81ee9446cf9014a2e695ef03e7` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:44:00.683709 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031d8bea-cd16-492e-91f1-cf0d0abc5b2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=49b76d81ee9446cf9014a2e695ef03e7)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:44:00.685655 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031d8bea-cd16-492e-91f1-cf0d0abc5b2e admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:44:00.732109 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-031d8bea-cd16-492e-91f1-cf0d0abc5b2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49b76d81ee9446cf9014a2e695ef03e7'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:44:00.740387 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-031d8bea-cd16-492e-91f1-cf0d0abc5b2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49b76d81ee9446cf9014a2e695ef03e7'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:44:00.740997 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-031d8bea-cd16-492e-91f1-cf0d0abc5b2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49b76d81ee9446cf9014a2e695ef03e7'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:44:00.753751 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-031d8bea-cd16-492e-91f1-cf0d0abc5b2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49b76d81ee9446cf9014a2e695ef03e7'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:44:00.760726 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-031d8bea-cd16-492e-91f1-cf0d0abc5b2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49b76d81ee9446cf9014a2e695ef03e7'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:44:00.767134 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1291/2590] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:44:00 2022] DELETE /identity/v3/users/49b76d81ee9446cf9014a2e695ef03e7 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:44:00.939394 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367c1169-a2fa-4e62-b43c-998cc8f83fb1 service neutron] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:44:00.949415 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367c1169-a2fa-4e62-b43c-998cc8f83fb1 service neutron] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:44:01.008419 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee8b652-8637-4d1e-9bf7-462c3d12c0c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:44:01.009051 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee8b652-8637-4d1e-9bf7-462c3d12c0c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:44:01.009335 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee8b652-8637-4d1e-9bf7-462c3d12c0c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:44:01.009609 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee8b652-8637-4d1e-9bf7-462c3d12c0c6 admin admin] PATH_INFO: `/v3/projects/8ae8c589d0b84ce096451a10717fc601` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:44:01.010713 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee8b652-8637-4d1e-9bf7-462c3d12c0c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ae8c589d0b84ce096451a10717fc601)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:44:01.012647 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee8b652-8637-4d1e-9bf7-462c3d12c0c6 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:44:01.061473 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-9ee8b652-8637-4d1e-9bf7-462c3d12c0c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ae8c589d0b84ce096451a10717fc601', 'request_id': 'req-9ee8b652-8637-4d1e-9bf7-462c3d12c0c6'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:44:01.067488 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-9ee8b652-8637-4d1e-9bf7-462c3d12c0c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ae8c589d0b84ce096451a10717fc601', 'request_id': 'req-9ee8b652-8637-4d1e-9bf7-462c3d12c0c6'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:44:01.074613 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-9ee8b652-8637-4d1e-9bf7-462c3d12c0c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ae8c589d0b84ce096451a10717fc601', 'request_id': 'req-9ee8b652-8637-4d1e-9bf7-462c3d12c0c6'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:44:01.084821 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-9ee8b652-8637-4d1e-9bf7-462c3d12c0c6 admin admin] The token cache is being invalidate because project 8ae8c589d0b84ce096451a10717fc601 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:44:01.085110 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-9ee8b652-8637-4d1e-9bf7-462c3d12c0c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:44:01.085733 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-9ee8b652-8637-4d1e-9bf7-462c3d12c0c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:44:01.087219 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1300/2591] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:44:00 2022] DELETE /identity/v3/projects/8ae8c589d0b84ce096451a10717fc601 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:44:01.091140 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031d8bea-cd16-492e-91f1-cf0d0abc5b2e admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:44:01.093170 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-031d8bea-cd16-492e-91f1-cf0d0abc5b2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:44:01.105963 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031d8bea-cd16-492e-91f1-cf0d0abc5b2e admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:44:01.162397 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbddf303-0411-43c5-ad4d-a247eed9ec20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:44:01.162967 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbddf303-0411-43c5-ad4d-a247eed9ec20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:44:01.163273 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbddf303-0411-43c5-ad4d-a247eed9ec20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:44:01.163544 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbddf303-0411-43c5-ad4d-a247eed9ec20 admin admin] PATH_INFO: `/v3/users/7e800e81836347afb340983d74b26b1c` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:44:01.164650 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbddf303-0411-43c5-ad4d-a247eed9ec20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e800e81836347afb340983d74b26b1c)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:44:01.166580 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbddf303-0411-43c5-ad4d-a247eed9ec20 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:44:01.210674 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-cbddf303-0411-43c5-ad4d-a247eed9ec20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e800e81836347afb340983d74b26b1c'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:44:01.218905 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-cbddf303-0411-43c5-ad4d-a247eed9ec20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e800e81836347afb340983d74b26b1c'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:44:01.219531 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-cbddf303-0411-43c5-ad4d-a247eed9ec20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e800e81836347afb340983d74b26b1c'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:44:01.231024 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-cbddf303-0411-43c5-ad4d-a247eed9ec20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e800e81836347afb340983d74b26b1c'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:44:01.233103 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:44:01.240043 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-cbddf303-0411-43c5-ad4d-a247eed9ec20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e800e81836347afb340983d74b26b1c'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:44:01.246207 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1292/2592] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:44:01 2022] DELETE /identity/v3/users/7e800e81836347afb340983d74b26b1c => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:44:01.415530 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee8b652-8637-4d1e-9bf7-462c3d12c0c6 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:44:01.416913 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-9ee8b652-8637-4d1e-9bf7-462c3d12c0c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:44:01.430584 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee8b652-8637-4d1e-9bf7-462c3d12c0c6 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:44:01.450794 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:44:01.491422 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71edb6c-2cb2-4661-a4e1-f29801ebd6ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:44:01.492051 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71edb6c-2cb2-4661-a4e1-f29801ebd6ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:44:01.492354 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71edb6c-2cb2-4661-a4e1-f29801ebd6ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:44:01.492649 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71edb6c-2cb2-4661-a4e1-f29801ebd6ff admin admin] PATH_INFO: `/v3/projects/164b302bf1bc417a9fd2affa94c91d40` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:44:01.493747 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71edb6c-2cb2-4661-a4e1-f29801ebd6ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=164b302bf1bc417a9fd2affa94c91d40)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:44:01.495663 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71edb6c-2cb2-4661-a4e1-f29801ebd6ff admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:44:01.542701 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-f71edb6c-2cb2-4661-a4e1-f29801ebd6ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '164b302bf1bc417a9fd2affa94c91d40', 'request_id': 'req-f71edb6c-2cb2-4661-a4e1-f29801ebd6ff'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:44:01.549206 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-f71edb6c-2cb2-4661-a4e1-f29801ebd6ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '164b302bf1bc417a9fd2affa94c91d40', 'request_id': 'req-f71edb6c-2cb2-4661-a4e1-f29801ebd6ff'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:44:01.557636 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-f71edb6c-2cb2-4661-a4e1-f29801ebd6ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '164b302bf1bc417a9fd2affa94c91d40', 'request_id': 'req-f71edb6c-2cb2-4661-a4e1-f29801ebd6ff'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:44:01.568362 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-f71edb6c-2cb2-4661-a4e1-f29801ebd6ff admin admin] The token cache is being invalidate because project 164b302bf1bc417a9fd2affa94c91d40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:44:01.568652 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-f71edb6c-2cb2-4661-a4e1-f29801ebd6ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:44:01.569269 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-f71edb6c-2cb2-4661-a4e1-f29801ebd6ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:44:01.570749 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1301/2593] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:44:01 2022] DELETE /identity/v3/projects/164b302bf1bc417a9fd2affa94c91d40 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:44:11.242737 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=4 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:44:11.565319 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:44:15.889637 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbddf303-0411-43c5-ad4d-a247eed9ec20 admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:44:15.890762 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-cbddf303-0411-43c5-ad4d-a247eed9ec20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:44:16.006417 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbddf303-0411-43c5-ad4d-a247eed9ec20 admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:44:16.096800 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ea1bad-3304-402b-8fa4-a518df5b25e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99889b1dab0b471d9c77887cffefc652', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:44:16.097472 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ea1bad-3304-402b-8fa4-a518df5b25e9 service nova] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:44:16.097753 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ea1bad-3304-402b-8fa4-a518df5b25e9 service nova] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:44:16.098022 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ea1bad-3304-402b-8fa4-a518df5b25e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:44:16.100018 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-17ea1bad-3304-402b-8fa4-a518df5b25e9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:44:16.113284 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ea1bad-3304-402b-8fa4-a518df5b25e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:44:16.115335 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ea1bad-3304-402b-8fa4-a518df5b25e9 service nova] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:44:16.126798 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1293/2594] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:44:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:44:23.159439 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f42a0f1-7274-4810-bbd5-454ba9c8acda None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:44:23.159919 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f42a0f1-7274-4810-bbd5-454ba9c8acda None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:44:23.160024 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f42a0f1-7274-4810-bbd5-454ba9c8acda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:44:23.208495 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-8f42a0f1-7274-4810-bbd5-454ba9c8acda None None] MFA Rules not processed for user `bf4faacf7df248d28f32d7014dadcc07`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 31 12:44:23.209687 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-8f42a0f1-7274-4810-bbd5-454ba9c8acda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:44:23.272059 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1302/2595] 10.222.0.15 () {58 vars in 967 bytes} [Mon Oct 31 12:44:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 31 12:44:23.312639 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ea1bad-3304-402b-8fa4-a518df5b25e9 service nova] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:44:23.314064 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-17ea1bad-3304-402b-8fa4-a518df5b25e9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:44:23.403140 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ea1bad-3304-402b-8fa4-a518df5b25e9 service nova] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:44:23.408925 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51d71a0-c829-4282-a49c-3584f3920958 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2254a5786c249b4aa79244f7198ae65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:44:23.409524 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51d71a0-c829-4282-a49c-3584f3920958 service swift] REQUEST_METHOD: `GET` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:44:23.409801 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51d71a0-c829-4282-a49c-3584f3920958 service swift] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:44:23.410070 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51d71a0-c829-4282-a49c-3584f3920958 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:44:23.414968 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=100 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:44:23.419348 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51d71a0-c829-4282-a49c-3584f3920958 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:44:23.421494 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51d71a0-c829-4282-a49c-3584f3920958 service swift] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:44:23.429428 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1294/2596] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Oct 31 12:44:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 31 12:44:30.062250 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f42a0f1-7274-4810-bbd5-454ba9c8acda None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:44:30.063982 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-8f42a0f1-7274-4810-bbd5-454ba9c8acda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:44:30.129902 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f42a0f1-7274-4810-bbd5-454ba9c8acda None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:44:30.134421 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081af200-57e4-4908-9fc1-c3f804b53cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:44:30.135046 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081af200-57e4-4908-9fc1-c3f804b53cdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:44:30.135353 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081af200-57e4-4908-9fc1-c3f804b53cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:44:30.135627 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081af200-57e4-4908-9fc1-c3f804b53cdc admin admin] PATH_INFO: `/v3/users/5ddf1d02fa2644d280d9dee5b3101366` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:44:30.136753 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081af200-57e4-4908-9fc1-c3f804b53cdc admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ddf1d02fa2644d280d9dee5b3101366)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:44:30.138689 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081af200-57e4-4908-9fc1-c3f804b53cdc admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:44:30.187917 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-081af200-57e4-4908-9fc1-c3f804b53cdc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ddf1d02fa2644d280d9dee5b3101366'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:44:30.194802 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-081af200-57e4-4908-9fc1-c3f804b53cdc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ddf1d02fa2644d280d9dee5b3101366'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:44:30.195480 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-081af200-57e4-4908-9fc1-c3f804b53cdc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ddf1d02fa2644d280d9dee5b3101366'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:44:30.196138 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-081af200-57e4-4908-9fc1-c3f804b53cdc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ddf1d02fa2644d280d9dee5b3101366'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:44:30.209023 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-081af200-57e4-4908-9fc1-c3f804b53cdc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ddf1d02fa2644d280d9dee5b3101366'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:44:30.210941 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:44:30.223002 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1303/2597] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:44:30 2022] DELETE /identity/v3/users/5ddf1d02fa2644d280d9dee5b3101366 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:44:30.401169 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51d71a0-c829-4282-a49c-3584f3920958 service swift] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:44:30.411779 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51d71a0-c829-4282-a49c-3584f3920958 service swift] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:44:30.467291 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd5b133-2b19-49a0-87d5-8759aebe634c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:44:30.467855 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd5b133-2b19-49a0-87d5-8759aebe634c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:44:30.468129 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd5b133-2b19-49a0-87d5-8759aebe634c admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:44:30.468397 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd5b133-2b19-49a0-87d5-8759aebe634c admin admin] PATH_INFO: `/v3/projects/0637756aceaf4644bf263b5b5d72d9ac` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:44:30.469474 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd5b133-2b19-49a0-87d5-8759aebe634c admin admin] RBAC: Authorizing `identity:delete_project(project_id=0637756aceaf4644bf263b5b5d72d9ac)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:44:30.471375 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd5b133-2b19-49a0-87d5-8759aebe634c admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:44:30.516976 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-ecd5b133-2b19-49a0-87d5-8759aebe634c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0637756aceaf4644bf263b5b5d72d9ac', 'request_id': 'req-ecd5b133-2b19-49a0-87d5-8759aebe634c'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:44:30.522302 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-ecd5b133-2b19-49a0-87d5-8759aebe634c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0637756aceaf4644bf263b5b5d72d9ac', 'request_id': 'req-ecd5b133-2b19-49a0-87d5-8759aebe634c'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:44:30.529491 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-ecd5b133-2b19-49a0-87d5-8759aebe634c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0637756aceaf4644bf263b5b5d72d9ac', 'request_id': 'req-ecd5b133-2b19-49a0-87d5-8759aebe634c'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:44:30.540263 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-ecd5b133-2b19-49a0-87d5-8759aebe634c admin admin] The token cache is being invalidate because project 0637756aceaf4644bf263b5b5d72d9ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:44:30.540543 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-ecd5b133-2b19-49a0-87d5-8759aebe634c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:44:30.541154 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-ecd5b133-2b19-49a0-87d5-8759aebe634c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:44:30.542609 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1295/2598] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:44:30 2022] DELETE /identity/v3/projects/0637756aceaf4644bf263b5b5d72d9ac => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:44:30.547334 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081af200-57e4-4908-9fc1-c3f804b53cdc admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:44:30.548874 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-081af200-57e4-4908-9fc1-c3f804b53cdc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:44:30.561988 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081af200-57e4-4908-9fc1-c3f804b53cdc admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:44:30.616977 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fe6021-ba66-468b-ab61-89937bd5664a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:44:30.617598 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fe6021-ba66-468b-ab61-89937bd5664a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:44:30.617873 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fe6021-ba66-468b-ab61-89937bd5664a admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:44:30.618140 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fe6021-ba66-468b-ab61-89937bd5664a admin admin] PATH_INFO: `/v3/users/789ac79c43df4cbab581ceada876d4e5` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:44:30.619309 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fe6021-ba66-468b-ab61-89937bd5664a admin admin] RBAC: Authorizing `identity:delete_user(user_id=789ac79c43df4cbab581ceada876d4e5)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:44:30.621180 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fe6021-ba66-468b-ab61-89937bd5664a admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:44:30.666244 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-e8fe6021-ba66-468b-ab61-89937bd5664a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '789ac79c43df4cbab581ceada876d4e5'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:44:30.673564 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-e8fe6021-ba66-468b-ab61-89937bd5664a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '789ac79c43df4cbab581ceada876d4e5'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:44:30.674169 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-e8fe6021-ba66-468b-ab61-89937bd5664a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '789ac79c43df4cbab581ceada876d4e5'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:44:30.674785 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-e8fe6021-ba66-468b-ab61-89937bd5664a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '789ac79c43df4cbab581ceada876d4e5'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:44:30.687494 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-e8fe6021-ba66-468b-ab61-89937bd5664a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '789ac79c43df4cbab581ceada876d4e5'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:44:30.699458 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1304/2599] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Oct 31 12:44:30 2022] DELETE /identity/v3/users/789ac79c43df4cbab581ceada876d4e5 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:44:30.865088 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd5b133-2b19-49a0-87d5-8759aebe634c admin admin] Authenticating user token {{(pid=81492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:44:30.866644 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.fernet_utils [None req-ecd5b133-2b19-49a0-87d5-8759aebe634c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:44:30.884805 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd5b133-2b19-49a0-87d5-8759aebe634c admin admin] Validating token access rules against request {{(pid=81492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:44:30.942537 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bccebfc-3acc-4e91-a829-e750d3aa9884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ddf15ba0c7488b8e49f974bba93de7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20cfeec04f5e4394b2ab5ceaa7d9c63a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:44:30.943155 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bccebfc-3acc-4e91-a829-e750d3aa9884 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:44:30.943460 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bccebfc-3acc-4e91-a829-e750d3aa9884 admin admin] SCRIPT_NAME: `/identity` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:44:30.943729 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bccebfc-3acc-4e91-a829-e750d3aa9884 admin admin] PATH_INFO: `/v3/projects/0be1ace5cc7842cd98dfc975e23f13fb` {{(pid=81492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:44:30.944821 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bccebfc-3acc-4e91-a829-e750d3aa9884 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0be1ace5cc7842cd98dfc975e23f13fb)` {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:44:30.946682 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bccebfc-3acc-4e91-a829-e750d3aa9884 admin admin] RBAC: Authorization granted {{(pid=81492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:44:30.977043 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:44:30.996212 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-5bccebfc-3acc-4e91-a829-e750d3aa9884 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0be1ace5cc7842cd98dfc975e23f13fb', 'request_id': 'req-5bccebfc-3acc-4e91-a829-e750d3aa9884'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:44:31.002250 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-5bccebfc-3acc-4e91-a829-e750d3aa9884 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0be1ace5cc7842cd98dfc975e23f13fb', 'request_id': 'req-5bccebfc-3acc-4e91-a829-e750d3aa9884'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:44:31.009476 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-5bccebfc-3acc-4e91-a829-e750d3aa9884 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0be1ace5cc7842cd98dfc975e23f13fb', 'request_id': 'req-5bccebfc-3acc-4e91-a829-e750d3aa9884'} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:44:31.020680 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-5bccebfc-3acc-4e91-a829-e750d3aa9884 admin admin] The token cache is being invalidate because project 0be1ace5cc7842cd98dfc975e23f13fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 31 12:44:31.020961 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-5bccebfc-3acc-4e91-a829-e750d3aa9884 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:44:31.021591 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG keystone.notifications [None req-5bccebfc-3acc-4e91-a829-e750d3aa9884 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 31 12:44:31.023056 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: [pid: 81492|app: 0|req: 1296/2600] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Oct 31 12:44:30 2022] DELETE /identity/v3/projects/0be1ace5cc7842cd98dfc975e23f13fb => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 31 12:44:40.696049 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=57,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:44:41.017758 dv-cinder-796170-6-iscsi devstack@keystone.service[81492]: DEBUG dbcounter [-] [81492] Writing DB stats keystone:DELETE=6,keystone:SELECT=14,keystone:INSERT=1 {{(pid=81492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 31 12:46:08.022964 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fe6021-ba66-468b-ab61-89937bd5664a admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 31 12:46:08.024571 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-e8fe6021-ba66-468b-ab61-89937bd5664a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:46:08.094531 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fe6021-ba66-468b-ab61-89937bd5664a admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 31 12:46:08.152016 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9532b059-7cae-4424-984e-8bf91287bedd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e7777495d7a406c8c50b1ad52e47263', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c32444cef584bce966b38203ce128b2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 31 12:46:08.152620 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9532b059-7cae-4424-984e-8bf91287bedd service placement] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 31 12:46:08.152894 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9532b059-7cae-4424-984e-8bf91287bedd service placement] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 31 12:46:08.153165 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9532b059-7cae-4424-984e-8bf91287bedd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 31 12:46:08.154854 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-9532b059-7cae-4424-984e-8bf91287bedd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 31 12:46:08.170685 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9532b059-7cae-4424-984e-8bf91287bedd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 31 12:46:08.172289 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9532b059-7cae-4424-984e-8bf91287bedd service placement] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 31 12:46:08.182879 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1305/2601] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Oct 31 12:46:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 31 12:46:18.178653 dv-cinder-796170-6-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=58 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}